System for imaging lesions aligning tissue surfaces

ABSTRACT

Methods, compositions and systems are provided for the imaging of cavity/tissue lesions, including without limitation cavity/tissue malignant lesions, e.g. cancers of the skin, mouth, colon, digestive system cervix, bladder, lung, etc.

CROSS REFERENCE

This application claims benefit and is a continuation of applicationSer. No. 14/364,053 filed Jun. 9, 2014, which is a 371 application andclaims the benefit of PCT Application No. PCT/US2012/071246, filed Dec.21, 2012, which claims benefit of U.S. Provisional Patent ApplicationNo. 61/578,441, filed Dec. 21, 2011, which applications are incorporatedherein by reference in their entirety.

FIELD OF THE INVENTION

The field of this invention is medical diagnostic devices and methods.More specifically, imaging, analysis and diagnosis of lesions aligningtissue surfaces, such as cancers of the skin, mouth, colon, digestivesystem cervix, bladder, lung, etc.

BACKGROUND

Cancer is a leading cause of death worldwide and accounted for 7.6million deaths (around 13% of all deaths) in 2008. Melanoma is anexample of a cancer aligning the skin tissue surface and used here as anexample. About 10% of all people with melanoma have a family history ofmelanoma. One is at increased risk of developing melanoma if there is afamily history of melanoma in one or more of your first-degree relatives(parent, brother or sister, or child).

Melanoma is currently the sixth most common cancer in American men andthe seventh most common in American women. The median age at diagnosisis between 45 and 55, although 25% of cases occur in individuals beforeage 40. It is the second most common cancer in women between the ages of20 and 35, and the leading cause of cancer death in women ages 25 to 30.

Melanoma is the most aggressive form of skin cancer. If it is recognizedand treated early it is almost always curable, but if it is not, thecancer can advance and spread to other parts of the body, where itbecomes hard to treat and can be fatal. While it is not the most commonof the skin cancers, it causes the most deaths. The American CancerSociety estimates that at present, about 120,000 new cases of melanomain the US are diagnosed in a year. In 2010, about 68,130 of these wereinvasive melanomas, with about 38,870 in males and 29, 260 in women.

There are four basic types of melanoma which differ in frequency andlocation in the body. All melanomas pose the same level of risk, basedon the following factors: Tumor depth (Breslow depth), Mitotic index(cells that are dividing within the melanoma), presence or absence ofulceration, number of regional lymph nodes containing melanoma, andextent of cancer spread in the regional lymph nodes.

Superficial spreading melanoma is the most common type of melanoma,representing about 70% of all cases. As its name suggests, it spreadsalong the epidermis for a period of months to years before penetratingmore deeply into the skin. The melanoma appears as a flat or barelyraised lesion, often with irregular borders and variations in color.Lesions most commonly appear on the trunks of men, the legs of women,and the upper back of both sexes. The earliest sign of a new superficialspreading melanoma is darkening in one part of a pre¬existing mole orthe appearance of a new mole on unaffected normal skin.

Nodular melanoma represents 15 to 30% of all melanomas. It grows deepermore quickly than other types of melanoma, and is found most often onthe trunk or head and neck. The melanoma usually appears as ablue-black, dome-shaped nodule, although 5% of lesions are pink or red.Nodular melanoma is more common in men than women.

Lentigo maligna melanoma arises from a pre-existing lentigo, rather thana mole, and accounts for approximately 5% of all melanoma cases. Thistype of melanoma typically takes many years to develop. It occurs mostoften in older adults, usually on the face and other chronicallysun-exposed areas. These melanomas are generally large, flat,tan-colored lesions containing differing shades of brown, or as in othermelanomas, black, blue, red, gray, or white.

Acral lentiginous melanoma accounts for less than 5% of all melanomasbut is the most common melanoma in African Americans and Asians;although this may also occur in light-skinned (Caucasian) individuals.The disease typically appears on the palms, soles, or under the nails.Lesions are usually tan, brown, or black, with variations in color andirregular borders. Because of the misconceptions that melanomas onlyoccur in sun-exposed areas, and that dark-skinned and Asian people arenot at risk for melanoma, these melanomas are often discovered laterthan other forms of melanoma. A tendency to mistake the early signs ofacral lentiginous melanoma for bruises or injuries to the palms, soles,or nailbeds may further delay diagnosis.

Early detection of melanoma is critical for treatment and survival. Whenmelanoma is found and treated early, the chances for long-term survivalare excellent. Five-year survival rates for patients with early-stage(Stage I) melanoma exceed 90 to 95%. As melanoma progresses, it becomesincreasingly more devastating and deadly. In later-stage disease, 5-yearsurvival rates drop to less than 85%. With early detection, survivalrates have improved steadily in recent years, and 85% of diagnosedpatients enjoy long-term survival after simple tumor surgery.

The first sign of melanoma is often a change in the size, shape, orcolor of an existing mole or the appearance of a new mole. Since thevast majority of primary melanomas are visible on the skin, there is agood chance of detecting the disease in its early stages. However,changes in size, shape, or color of an existing mole or the appearanceof a new mole is not always conclusive of presence of melanoma in amole. Men most commonly develop melanoma on the trunk, particularly theback, and women on the legs or arms.

If the primary care physician suspects one may have melanoma, one willbe referred to a dermatologist, a medical oncologist, or a surgicaloncologist. To make a definitive diagnosis, they will performexaminations and tests. The doctor will first take a complete medicalhistory to learn about ones symptoms and risk factors. Your age, timesince your first concern, changes in features, sun burns, family historyof atypical moles or skin cancer, particularly melanoma. Complete skinexamination. Dermoscopy, Biopsy, Lymph node examination, chest x-ray, CTscan, Magnetic resonance imaging (MRI), Serum lactate dehydrogenase(LDH).

Melanoma is staged is based on the risk factors most important indetermining prognosis. They include: Tumor thickness (also known asBreslow thickness): how deeply the tumor has penetrated the skin.Thickness is measured in millimeters (mm). Thinner tumors carry a morefavorable prognosis than thicker tumors. The thicker the tumor, thegreater the risk of tumor metastasis. The presence or absence of tumorulceration: A condition in which the epidermis that covers a portion ofthe primary melanoma is not intact. Ulcerated tumors pose a greater riskfor metastatic disease than tumors that are not ulcerated. Mitoses:Active cell division of the tumor and can be defined in terms of number.This is determined by the pathologist. The more mitoses, the moreaggressive the tumor growth. Metastatic lymph nodes. The greater thenumber of lymph nodes containing melanoma, the less favorable theprognosis. Whether metastasis to the lymph nodes is microscopic ormacroscopic. Micrometastases are tiny tumors. They can be detected onlyby microscopic evaluation after sentinel lymph node biopsy or electivelymph node dissection. Macrometastases can be felt during physicalexamination or seen by the naked eye when inspected by a surgeon orpathologist. Their presence is confirmed by lymph node dissection orwhen the tumor is seen to extend beyond the lymph node capsule.Macrometastases carry a less favorable prognosis than micrometastases.The site of distant metastasis. Distant metastases to the skin, thesubcutaneous tissue, or distant lymph nodes carry a relatively betterprognosis than distant metastases to any other site in the body. Levelof serum lactate dehydrogenase (LDH). LDH is an enzyme found in theblood and many body tissues. Elevated LDH levels usually indicate thepresence of metastatic disease—and a less favorable prognosis thannormal LDH levels.

The TNM Staging System was created by the American Joint Committee onCancer (AJCC). The system defines cancer stage by describing:

T: the features of the primary tumor. The three distinguishing featuresare tumor thickness, mitoses, and ulceration. Tumor thickness (alsoknown as Breslow depth) is measured in millimeters (mm).

N: the presence or absence of tumor spread to nearby lymph nodes

M: the presence or absence of metastasis to distant sites

Revised TNM Classification

Abbreviations: N/A, not applicable; LDH, lactate dehydrogenases.

T Classification Thickness Ulceration Status Tis N/A N/A T1 ≤1.0 mm a:w/o ulceration and mitosis < 1/mm² b: with ulceration and mitosis ≥1/mm² T2 1.01-2.0 mm a: w/o ulceration b: with ulceration T3 2.01-4.0 mma: w/o ulceration b: with ulceration T4 >4.0 mm a: w/o ulceration b:with ulceration N Classification # of Metastatic Nodes Nodal MetastaticMass N0 No evidence of lymph node metastasis   1 node a: micrometastasisN1 b: macrometastasis 2-3 nodes a: micrometastasis b: macrometastasis N2c: In transit metastases/satellites without metastatic nodes N3 4 ormore metastatic nodes, or matted nodes, or in-transitmetastases/satellites and metastatic nodes M Classification Site SerumLDH M0 No evidence of metastasis to distant tissues or organs M1aDistant skin, subcutaneous or nodal Normal metastases M1b Lungmetastases Normal M1c All other visceral metastases Normal Or anydistant metastases Elevated

One of the most important factors in staging melanoma—and in determiningtreatment and prognosis—is how deeply the tumor has penetrated the skin.Tumor depth is described in two ways: Breslow thickness is a method ofmeasuring how deeply the primary tumor has penetrated the skin,regardless of anatomic layer. Tumor penetration is measured inmillimeters (mm) from the epidermis to the deepest point of penetration.(1.0 mm=0.04 inch, or less than 1/16 inch.) Breslow thickness hasreplaced Clark level as a more accurate measurement of tumor depth andmore predictive of prognosis. The thicker the tumor, the greater thechance it has metastasized to regional lymph nodes or distant sites.

Clark level is a method of measuring how deeply the primary tumor haspenetrated the skin based on anatomic layer. The deeper the layer ofpenetration, the greater the chance the tumor has metastasized toregional lymph nodes or distant sites. Since skin thickness variesthroughout the body, Clark level is considered to be less accurate thanBreslow thickness in describing tumor penetration. In fact, in the newAmerican Joint Committee on Cancer (AJCC) staging system for melanoma,Clark level is no longer considered a secondary characteristic of StageI tumors no more than 1.0 mm thick. This has been replaced with mitoses.

Clark level I. The tumor is located only in the lowest layer of theepidermis, known as the dermo-epidermal junction. Level I is also knownas melanoma in situ. Clark level II. The tumor has partially penetratedthe papillary dermis, the loose connective tissue beneath the epidermis.Clark level III. The tumor has completely penetrated and filled thepapillary dermis. Clark level IV. The tumor has penetrated through thepapillary dermis to the dense connective tissue of the reticular dermis.Clark level V. The tumor has penetrated through the reticular dermis tothe subcutaneous tissue, the fatty layer beneath the skin.

Melanoma is now grouped into the following stages according to therevised TNM staging system:

Stage 0 melanoma involves the epidermis but has not reached theunderlying dermis. This stage is also called melanoma in situ (TisNOMO).Stage 0 melanoma is very early stage disease known as melanoma in situ(Latin for “in place”). Patients with melanoma in situ are classified asTis (tumor in situ). The tumor is limited to the epidermis with noinvasion of surrounding tissues, lymph nodes, or distant sites. Melanomain situ is considered to be very low risk for disease recurrence orspread to lymph nodes or distant sites.

Stage I melanoma is characterized by tumor thickness, presence andnumber of mitoses, and ulceration status. There is no evidence ofregional lymph node or distant metastasis.

There are two subclasses of Stage I melanoma.

Stage IA: T1aNOMO (tumor less than or equal to 1 mm, no ulceration, andno mitoses). Stage IB: T1 bNOMO or T2aNOMO (tumor less than or equal to1 mm, with ulceration or mitoses).

Stage I melanomas are localized tumors. This means the primary tumor hasnot spread to nearby lymph nodes or distant sites. Stage I melanomas areconsidered to be low-risk for recurrence and metastasis.

Stage I melanomas are defined by two primary characteristics:

Tumor thickness (known as Breslow depth): how deeply the tumor haspenetrated the skin. Thickness is measured in millimeters (mm).Ulceration: a condition in which the epidermis that covers a portion ofthe primary melanoma is not intact. Ulceration is determined bymicroscopic evaluation of the tissue by a pathologist, not by what canbe seen with the naked eye. Mitoses: A condition of the cells being in astate of active division. Mitoses are determined by microscopicevaluation by a pathologist, not what can be seen with the naked eye,similar to ulceration. It will be defined as “present or not present”and should include a number of mitoses per mm2. The designation of Clarklevel measures the depth of invasion according to the number of layersof skin the tumor has penetrated. There are five anatomic layers of theskin: Level I: epidermis. Levels 11-IV: dermis. Level V: the subcutis.

Clark level is no longer considered by the new American Joint Committeeon Cancer (AJCC) staging system for melanoma, as a secondarycharacteristic of Stage I tumors no more than 1.0 mm thick. This hasbeen replaced with mitoses.

Subclasses of Stage I Melanoma

Stage IA (T1 aNOMO) T1a: the tumor is no more than 1.0 millimeter (mm)thick, with no ulceration and no mitoses. NO: the tumor has not spreadto nearby lymph nodes. MO: the tumor has not spread to sites distantfrom the primary tumor. Stage IB (T1 bNOMO or T2aNOMO). T1b: the tumoris no more than 1.0 mm thick, with ulceration or presence of >1 mitoses.T2a: the tumor is 1.01-2.0 mm thick, with no ulceration. NO: the tumorhas not spread to nearby lymph nodes. MO: the tumor has not spread tosites distant from the primary tumor.

Stage II melanoma is also characterized by tumor thickness andulceration status. There is no evidence of regional lymph node ordistant metastasis.

There are three subclasses of Stage ii melanoma.

Stage MA: T2bNOMO or T3aNOMO

Stage II B: T3bNOMO or T4aNOMO

Stage IIC: T4bNOMO

Stage III melanoma is characterized by the level of lymph nodemetastasis. There is no evidence of distant metastasis.

There are three subclasses of Stage ill melanoma.

Stage MIA: T1-T4a N1aMO or T1-T4aN2aMO

Stage IIIB: T1-T4bN 1aMO, T1-T4bN2aMO, T1-T4aN 1bMO, T 1-T4aN2bMO, orT1-T4a/bN2cMO

Stage MIC: T1-4bN 1bNO, T1-4bN2bMO, or T1-4a/bN3MO

Stage IV melanoma is characterized by the location of distant metastasesand the level of serum lactate dehydrogenase (LDH).

Stage IV melanomas include any T or N classification. For details, seeStage IV.

Treatments are available for all people with melanoma. In many cases,the standard treatment is surgery to remove the tumor and a surroundingarea of normal-appearing skin. Sometimes surgery is followed byadditional therapy such as immunotherapy, chemotherapy, radiation, or acombination of these treatments. Chemotherapy and immunotherapy are alsoused to treat advanced or recurrent melanoma.

Tumors need blood flow to grow bigger than 2-3 mm. Judah Folkman firstarticulated the importance of angiogenesis for tumor growth in 1971. Hestated that the growth of solid tumors remains restricted to 2-3 mm indiameter until the onset of angiogenesis. Tumors need oxygen andnutrients. For the first 2 mm of their growth (-one million cells)tumors get their oxygen and nutrients from the host capillaries andextracellular fluid. As they outgrow the host supply they start makingtheir own blood vessels. Cancers “persuade” the existing hostcapillaries to sprout, change direction and grow throughout the tumor.To do this, they secrete growth factors—angiogenic factors.

Angiogenesis (neoangiogenesis) is a multistep process, which isregulated by a balance between pro- and antiangiogenic factors.Microtumor foci remain dormant until a biological event occurs totrigger growth beyond the 2 mm stage/size. One trigger is aninsufficient nutrient supply resulting in hypoxic cells.State-of-the-art clinical PET scanners, are able to detect tumor fociwith a resolution of 3-4 mm. Preclinical animal scanners allow forresolutions in the 1 mm range in small rodents.

Vascularization in melanoma occurs and melanoma becomes metastatic(>0.75 mm). Human malignant melanoma is a highly metastatic tumor withpoor prognosis and high resistance to treatment. It progresses throughdifferent steps: nevocellular nevi, dysplastic nevi (when these twoentity can be identified as primary events in melanocytic neoplasiaprogression), in situ melanoma, radial growth phase melanoma (Breslowindex <0.75 mm), vertical growth phase melanoma (index >0.75 mm), andmetastatic melanoma. Breslow's depth is used as a prognostic factor inmelanoma of the skin. It is a description of how deeply tumor cells haveinvaded. Melanomas in the vertical growth stage phase are metastatic.

Primary melanoma tumor grows horizontally through the epidermis(non-invasive phase); over time, a vertical growth phase componentintervenes and melanoma increases its thickness and invades the dermis(invasive phase). Once a vertical growth phase has developed, there is adirect correlation between the tumor thickness and the number ofmetastases.

Blood flow occurs in melanoma index >0.8 mm. To correlate melanomathickness and angiogenesis, the blood flow in 71 primary skin melanomaswere investigated using a 10 MHz Doppler ultrasound flowmeter. Flowsignals were analyzed on an Angioscan-I I spectrum analyzer. Dopplerflow signals were detected in 44 tumors, with a close relationship toBreslow's tumor thickness. No blood flow signal was detected in 27lesions and 25 of these had a tumor thickness of 0.8 mm or less.Ninety-seven percent of tumors of thickness >0.8 mm had detectableDoppler flow signals. This study indicates the development of aneovascular bed as the tumor thickness approaches 0.8 mm. An additionalstudy of tumor blood flow in 36 patients, 38 with malignant melanomasusing Doppler Ultrasound flowmetry showed that tumor blood flow can bedetected in most melanomas more than 0.9 mm thick, and is absent in mostmelanomas less than this thickness

Cancer starts in a single cell. Cells accumulate genetic changes andbecome abnormal. During the early stages of tumor development first amicro tumor lesion is formed. At the second stage a tumor lesion isformed which expands beyond the size of the micro lesion. In the finalstage tumor cells are released to the circulatory system in the processof metastasis.

Tumors remain dormant as microfoci in the body. At the stage of a microtumor lesion, signals from the immune system can hold a micro tumor incheck in a state of tumor dormancy by the tumor inability to grow beyondit's local macroenvironment. In this state, the level of cellproliferation is in balance to the level of cell death. As tumorsaccumulate additional genetic changes, they are able to disrupt thisbalance and grow beyond microfoci and the macroenvironment. Thisbalanced state is overturned when the signals originating from the tumorincrease overpowering the signals from the immune system. Tumor cellssecret signaling proteins to the tumor microenvironment andmacroenvironment. During this stage emerging tumors signal to themacroenvironmet the need, to expand into additional space and additionalnutrient as well as oxygen required for expanded growth. These signalsprepare the tumor environment for expanded tumor growth (increasingspace) and for an increase in nutrient supply (angiogenesis). Theprocess is mediated by growth factors, cytokines and other activationsproteins released from the tumor cells or from the tumor macro/microenvironment during tumor expansion.

These processes can be investigated by testing the tissue surrounding atumor (macro environment-tissue surrounding the diseased tissue). Oftenthis environment may be difficult to study, especially if the tumor isembedded deep in a tissue. However, tumor growths in the proximity ofthe tissue cavity/surface compartment are good candidates to this typeof investigation. Examples of such tumors can be: skin cancers, mouthcancer, lung cancer, colon cancer, digestive system cancer, cervicalcancer, bladder cancer, etc.

The ideal medical diagnostic procedure and tools would have thefollowing characteristics: is minimally invasive or non-invasive;permits early stage disease detection; permits early body response to anew antigen; permits early body response to a foreign antigen, monitorsdisease development and progression; investigates the macro environmentof the diseased cell and/or tissue as indication of presence of disease;is easy to use, low cost, provides a quick test to perform; can beperformed by someone other than the physician; operates independent ofskin color or ethnicity; provides immediate test results, providesconsistency of results, works for a wide range of lesion types and bodylocations; is minimally dependent on human interpretation; is a simpletest—minimal training necessary; provides automated or machine-assistedmedical documentation, such as photographs or quantified test metrics;provides automated or machine-assisted electronic medical recordkeeping, such as machine readable codes on samples and files thatdirectly tie to patient, doctor and date; operates independent of visualcues such as, color, shape and size; can identify melanoma in amelanoticskin lesion; and/or can identify melanoma in small lesions, less than 5mm. Current technology has weaknesses in all or some of the above areas.

Also, the prior art uses an industrial camera connected to a computer.This arrangement is either impossible to hand hold, is too cumbersome torealistically handhold, or is difficult to consistently place in thecorrect position.

Prior art uses a fixed focal length lens, which only works when thecamera can be placed a fixed distance from the subject. When usingfluorescent biomarkers it is preferable to block all ambient and straylight from entering the optical path between the camera optics and thepatient's skin. Therefore, some kind of physical light shield or lightbaffle is employed. This light baffle is normally affixed to the camera,surrounding the lens with an approximately pyramidal or conical shape,with the truncated point of the pyramid/cone being at the camera and thebase of the pyramid/cone against the patient's skin. This approach issometimes adequate for relatively flat or convex areas of skin, such ason a patient's back. The fixed focal point of prior art is fixed at thedistance of the base of the pyramidal light shield. However, arrangementfails for some lesion locations, such as on the side of a patient's nosewhere the light baffle will not block the ambient or stray light.

Such art can include one or more of the following: Balch et al. J ClinOncol 2001; 19:3635-3648; Folkman, J. (1971). New England Journal ofMedicine, 285, 1182-1186; Folkman J, Klagsbrun M. In: Gottlieb A A,Plescia O J, Bishop D H L, eds. Fundamental Aspect of Neopla.sia.Berlin, Springer, 1975, 401-412; Ellis, et al. (2002). Oncology, 16,14-22; Carmeliet, P., & Jain, R. K. (2000). Nature, 407, 249-257;Matsumoto et al. (2006). Performance characteristics of a new3-dimensional continuous emission and spiral-transmission highsensitivity and high resolution PETcamera evaluated with the NEMA NU2-2001 standard. Journal of Nuclear Medicine, 47, 83-90; Chatziioannou,A. F. (2005). Instrumentation for molecular imaging in preclinicalresearch: Micro-PET and Micro-SPECT. Proceedings of the AmericanThoracic Society, 2, 533-536; Breslow, Annals of Surgery, vol. 172, no.5, pp. 902-908, 1970; Heasley, S. Toda, and M. C. Mihm Jr., SurgicalClinics of North America, vol. 76, no. 6, pp. 1223-1255, 1996;Srivastava A, Hughes L E, Woodcock J P, Laidler P. Vascularity incutaneous melanoma detected by Doppler sonography and histology:correlation with tumour behaviour. Br J Cancer. 1989 January;59(1):89-91; Srivastava A, Laidler P, Hughes L E, Woodcock J, Shedden EJ: Neovascularization in human cutaneous melanoma: A quantitativemorphological and Doppler ultrasound study. Eur J Cancer Clin Oncol1986, 22:1205-1209, which are hereby incorporated by reference in theirentirety.

SUMMARY OF THE INVENTION

A need exists for improved systems and methods having desirablediagnostic features. Such desirable diagnostic features may include oneor more of the following: is minimally invasive or non invasive; permitsearly stage disease detection; permits early body response to a newantigen; permits early body response to a foreign antigen, monitorsdisease development and progression; investigates the macro environmentof the diseased cell and/or tissue as indication of presence of disease;is easy to use, low cost, provides a quick test to perform; can beperformed by someone other than the physician; operates independent ofskin color or ethnicity; provides immediate test results, providesconsistency of results, works for a wide range of lesion types and bodylocations; is minimally dependent on human interpretation; is a simpletest—minimal training necessary; provides automated or machine-assistedmedical documentation, such as photographs or quantified test metrics;provides automated or machine-assisted electronic medical recordkeeping, such as machine readable codes on samples and files thatdirectly tie to patient, doctor and date; operates independent of visualcues such as, color, shape and size; can identify melanoma in amelanoticskin lesion; and/or can identify melanoma in small lesions, less than 5mm.

Methods, compositions and systems are provided for the imaging oflesions aligning tissue cavity/surfaces, including without limitationmalignant lesions aligning tissue cavity/surfaces, e.g. cancers of theskin, mouth, cervix, bladder, etc. In some embodiments, the methods finduse in the diagnosis of skin cancers including melanoma, skin basal cellcarcinoma, etc., and non-cancerous skin diseases. The methods of theinvention are useful in detection of melanoma not restricted by type,color, size, and body location or Breslow thickness and Clark level orthe ABCDEF criteria.

In one or more embodiments of the invention, a detectably labeled biotagis applied to a tissue surface of an individual, e.g. skin, oral mucosalsurface, bladder, cervix, lung, gastrointestinal and the like, generallyin the region of a suspected lesion. The biotag selectively binds to atargeted binding partner present in lesions of interest. Alternativelythe biotag is absorbed or metabolized or internalized or retained inother manner in reactive tissue. Application may be topical, for exampleapplication of a gel, liquid, etc. to the surface of the skin using askin penetration agent or facilitator, or may be applied by sub- orintradermal injection, e.g. with one or an array of microneedles, to adepth of up to 1-5 mm or by electrical conductivity.

In some embodiments the tissue surface is preconditioned to increasedelivery of the biotag through the surface, e.g. skin, etc.Preconditioning may include topical administration of a penetrationvehicle in the absence of the biotag, where the vehicle optionallycomprises a blocker agent, as described herein. The residual biotag isremoved from the surface after a period of time sufficient for thebiotag to penetrate the tissue surface.

A photograph of the tissue surface is taken using a camera and a lightof the right (excitation) wavelengths that activates the biotagdetectable label (in emission wavelengths) while taking a photographcapturing light of the same emission wavelength as the label. In someembodiments, the detectable label is a fluorescent label. When thelesion is diseased, e.g. cancerous; then the biotag will bind to thetargeted binding partner, for example a cancer marker. Such binding canoccur in the macroenvironment of the lesion, e.g. tumor vasculature, oron the cell surface, or within the diseased cells, and will be visiblein the photograph. If the target marker is absent, the label will besubstantially absent from the photograph (e.g., below background). Insome embodiments, the biotag can bind to markers present in themacroenvironment proximal to the tumor cell even when a diseased cell isnot specifically present in the area being photographed. Optionally animage is taken to establish the base line status for the specificpatient prior to application of the biotag.

The camera can also be used to take a picture of the same area usingvisible light. The two photos can be presented to a physician to analyzeand compare. In some instances, the photos can be dynamically overlaidso that the physician can see where the retention of the biotag occurson the tissue surface relative to features that are visually apparent,e.g. a mole, lesion, etc. Each photo can also be presented separately orcombined as an overlay.

The invention provides a solution to significant prior problems withimplementation. Prior art uses large, specialized expensive industrialcameras. Typical cameras used in the art cannot be practically or easilyhandheld. Variable focus at two different light wavelengths is a problemnot resolved in the prior art. This invention overcomes numerousweaknesses in the prior art, in multiple embodiments.

Embodiments of the invention provide advantageous features andcharacteristics in the areas of the biotag, the camera, imageidentification, and/or automated image analysis, including methods,systems and/or devices of manufacture.

Aspects of various embodiments may include one or more of the followingfeatures, capabilities, or results, as are listed first in the tablebelow, then explained in more detail.

TABLE 1 Use of a consumer, integrated camera as a starting point for themethod of manufacturing the camera, for low cost, ability to be handheld, compactness, portability, and reliability. An industrial cameracan also be used. Use of the camera's built-in autofocus capabilities,and the autofocus software may be changed or updated. Use of thecamera's autofocus in the infrared range. Use of the lens that comeswith the camera, either integrated with the body or an interchangeablelens designed for use with the camera, preferably a macro-lens. Two,three, or four light sources - one visible light for skin, oneexcitation light for cancer detection, one emission light for autofocus(option), and one for 3D and surface roughness image capture andanalysis (option). Dual, dynamically selectable light filters, one bandpass for visible, one for IR detection. Optionally, a single filter withtwo pass-bands and one-stop band may be used. Taking two photographs intwo different wavelength bands using the same camera, optics, controls,and image storage. Methods to align display and view the two abovephotographs. Use of a biotag in conjunction with the other features ofthis invention. Use of a multi-function fiducial. Use of aneight-function fiducial (exposure/brightness, calibration, focus,patient orientation, patient id, linear metric, alignment of multipleimages, number of mole on patient) optional information about thediagnostic procedure. Use of machine-readable codes (1 D, 2D, text) onthe fiducial, with machine printed on demand, pre-printed, hand-writtenareas, or a hybrid of these manufacturing attributes. Use of bothvisible fiducial features and emission spectra features on the same orseparate fiducial. Changing the autofocus firmware on the camera tohandle focusing in the infrared light band. Removing the infraredblocking filter originally placed in the primary optical path in thecamera during original manufacturing. Removing the infrared blockingfilter originally placed in the focusing optical path in the cameraduring original manufacturing. Removing the RGB filter on top of theinfrared filter optional. Removing the RGB filter will give betterfluorescent sensitivity (about 15% more light) and higher pixel count influorescent but visible image will be black and white and not color. Useof an engineered diffuser for one or two light sources. Integrating alllight sources and filters into a single, integrated camera that operateswithout an external computer or the need for external connectivity orpower.

This invention may include a consumer, prosumer, or “integrated” digitalcamera. This provides a large amount of value, functionality, andconvenience, compared to large, expensive, non-portable custom-mademedical cameras. Prior art has not been able to use an integrated camerain this application because of major deficiencies: the prior art cameracan neither image nor auto-focus in the infrared light band. Also, theprior art camera does not include the necessary light sources nor thenecessary filters. In addition, the prior art camera has no ability tooverlay two different images.

This invention overcomes all of these deficiencies yet still maintainsthe fundamental benefits of a low-cost, compact, portable, handheld,reliable camera. Systems and methods provided herein may advantageouslyinclude removing one, two or three filters originally manufactured inthe camera, modifying the auto-focus, adding light sources, addingdynamically selectable filters, and the use of complex fiducials toenable auto-exposure, auto-focus, and image alignment. When imaginglight below 700 or 710 or 720 or 730 or 740 or 750 nm the filter manynot need to be removed.

Autofocus is particularly important for two reasons. First, all straylight must be or is preferably blocked during exposure so that themaximum amount of light in the image is from the fluorescent biotag,along with information on the fiducial. This light blocking may beaccomplished by having a light baffle that extends from the camera topatient's skin. Preferably, the end of the baffle that touches thepatient is flexible to accommodate variations in the skin and thepatient's anatomy. However, this flexibility and these variations meanthat the distance from the area of interest to the camera lens is notconstant. Traditional fixed focus cameras lack this flexibility, eithercompromising exposure or requiring a smaller numerical aperture, whichlets in less light resulting in an inferior image or possibly blurringdue to the long exposure then required. An alternative to a baffle is toplace a cloth or other light blocking means over the patient and cameracombination. This makes autofocus even more critical as it is now harderand less convenient to implement a fixed distance between the cameralens and the patient's area of interest, now hidden under the cloth.Another option is to turn off all room lights, which is even moreimpractical as well as having at least the same drawbacks as the clothbaffle.

Second, the patient's anatomy may present the area of interest in arecess, such at the side of the nose, making it very difficult orimpossible to place the end of the camera baffle precisely at the rightdistance for a fixed focus camera.

Autofocus solves these practical problems of imaging a patient's skin inan environment free of stray light. However, implementing autofocus hasnot been achievable using prior art systems and techniques, which areovercome by this invention.

In another embodiment of this invention, features of the fiducials areused to enable automatic image analysis, processing, categorization,identification and filing of images.

Before the present compositions and methods are described in furtherdetail, it is to be understood that this invention is not limited toparticular methods described, as such may, of course, vary. It is alsoto be understood that the terminology used herein is for the purpose ofdescribing particular embodiments only, and is not intended to belimiting, since the scope of the present invention will be limited onlyby the appended claims.

Where a range of values is provided, it is understood that eachintervening value, to the tenth of the unit of the lower limit unlessthe context clearly dictates otherwise, between the upper and lowerlimit of that range and any other stated or intervening value in thatstated range is encompassed within the invention. The upper and lowerlimits of these smaller ranges may independently be included in thesmaller ranges, subject to any specifically excluded limit in the statedrange. As used herein and in the appended claims, the singular forms“a”, “and”, and “the” include plural referents unless the contextclearly dictates otherwise.

Unless defined otherwise, all technical and scientific terms used hereinhave the same meaning as commonly understood by one of ordinary skill inthe art to which this invention belongs. Although any methods andmaterials similar or equivalent to those described herein can also beused in the practice or testing of the present invention, preferablemethods and materials are now described. All publications, patents, andpatent applications mentioned herein in this specification areincorporated herein by reference to disclose and describe the methodsand/or materials in connection with which the publications are cited.Furthermore, all publications, patents, and patent applicationsmentioned in this specification are herein incorporated by reference tothe same extent as if each individual publication, patent, or patentapplication was specifically and individually indicated to beincorporated by reference.

The publications discussed herein are provided solely for theirdisclosure prior to the filing date of the present application. Nothingherein is to be construed as an admission that the present invention isnot entitled to antedate such publication by virtue of prior invention.Further, the dates of publication provided may be different from theactual publication dates, which may need to be independently confirmed.

Definitions

We provide below the definition of terms as used herein.

AF—Autofocus.

Attachment to integrated imaging device—The attachment of keycomponents, such as an excitation light source and/or filter, to thecamera body may be permanent or temporary; may use an intermediatestructure such as a plate or arm, clamp, lens, external battery pack orother external accessory to the camera, or other intermediate mechanicalmeans. Suitable attachment is demonstrated by having a useable medicalcamera assembly, which may be hand-held. Key components of the cameramay be supplied by and/or installed by the user, a technician, medicalprofessional or other person including: battery or external batterypack, memory card, illumination modules, lens, filter, light hood orother modular, separable, standardized or interchangeable components.Components of the invention may be offered as a kit or may come fromdifferent suppliers.

Barcode—Machine-readable printing information, including 1D or 2D barcodes, matrix codes, OCR fonts, QR codes, etc.

Biotag—A specific binding partner to a targeted molecule of interest.Examples of biotags include, without limitation, peptide,peptidomimetic, peptoid, circular peptide, etc.; a nucleic acid such asRNA, DNA, aptamer, etc.; or other organic compound. One, or a cocktailof biotags of 2, 3 4, or more different moieties may be used in themethods of the invention for multiplex imaging. The biotag is of amolecular weight small enough to effectively cross the epidermalsurface, e.g. usually less than 10,000 daltons, less than 5,000 daltons,less than 2,500 daltons, less than 1,000 daltons, which penetration maybe facilitated by a penetration agent. The biotag generally comprises adetectable label.

Molecules suitable as binding partners to a biotag of the inventioninclude, for example, cancer-associated markers present on cancer orpre-cancerous cells, or in the macroenvironment of cancerous orpre-cancerous cells, e.g. the vasculature at the site of the lesion.Specific markers of interest for this purpose include, withoutlimitation, molecules associated with tumor vasculature, such asintegrins, including integrin av, integrin a5, integrin β3, integrin β1,etc. Biotags suitable for detection of such integrins include peptidescomprising an RGD motif or mimetics thereof, as known and used in theart. See, for example, Gaertner et al. (2012) Eur J Nucl Med MolImaging. 39 Suppl 1:S1 26-38; Danhier et al. (2012) Mo. Pharm.9(11):2961-73, herein specifically incorporated by reference. Otherbiotags of interest include, without limitation, hormones, antigenbinding fragments of antibodies, EGF, IGF, etc.

Tumor-associated antigens may include, without limitation, immunogenicsequences from MART-1, gp100 (pmel-17), tyrosinase, tyrosinase-relatedprotein 1, tyrosinase-related protein 2, melanocyte-stimulating hormonereceptor, MAGE1, MAGE2, MAGE3, MAGE12, BAGE, GAGE, NY-ESO-1, β-catenin,MUM-1, CDK4, caspase 8, KIA 0205, HLA-A2R1701, α-fetoprotein, telomerasecatalytic protein, G-250, MUC-1, carcinoembryonic protein, p53,Her2/neu, triosephosphate isomerase, CDC-27, LDLR-FUT, telomerasereverse transcriptase, MUC18, ICAM-1, TNF α/β, plasminogen activator(uPA), Cathepsins (B, D, H, L), PSMA, HMB-45, S-100, Melan-A (A103),(T311), Mitf (D5), Glypican-3, GPC3, GPNMB, MIA (melanoma inhibitoryactivity), MCR-1, EGF, IGF, ARPC2, FN1, RGS1, SPP1, WNT2, PECAM-1,osteopontin, glucose, MMP-s (matrix metalloproteinase family memberssuch as MMP-1. MMP-2, MMP-9, MMP-13, MT I-MMP and others) FDG (or othermetabolites), VEGF, and the like, as known in the art.

Optically visible moieties for use as a detectable marker includefluorescent dyes, or visible-spectrum dyes, visible particles, and othervisible labeling moieties. Fluorescent dyes such as fluorescein,coumarin, rhodamine, bodipy Texas red, and cyanine dyes, are useful whensufficient excitation energy can be provided to the site to be inspectedvisually. Endoscopic visualization procedures may be more compatiblewith the use of such labels. Acceptable dyes include FDA-approved fooddyes and colors, which are non-toxic, although pharmaceuticallyacceptable dyes which have been approved for internal administration arepreferred. Alternatively, visible particles, such as colloidal goldparticles or latex particles, may be coupled to the biotag via asuitable chemical linker.

Fluorescent dyes of interest as a detectable label include, withoutlimitation, fluorescein, rhodamine, indocyanine green (ICG), Texas Red,phycoerythrin, allophycocyanin, 6-carboxyfluorescein (6-FAM),2′,7′-dimethoxy-4′,5′-dichloro-6-carboxyfluorescein (JOE),6-carboxy-X-rhodamine (ROX),6-carboxy-2′,4′,7′,4,7-hexachlorofluorescein (HEX), 5-carboxyfluorescein(5-FAM) or N,N,N′,N′-tetramethyl-6-carboxyrhodamine (TAMRA), the cyaninedyes, such as Cy3, Cy5, Cy 5.5, Alexa 542, Alexa 647, Alexa 680, Alexa700, Bodipy 630/650, fluorescent particles, fluorescent semiconductornanocrystals, and the like.

In some embodiments, the wavelength for emission from the label is inthe range of the near infrared. Such labels include, without limitation,Alexa dyes such as Alexa 647, Alexa 680, Alexa 700 and Cyanine dyes suchas Cy 5, Cy5.5. Cy7 Characteristics considered for label selectioninclude its light absorption, and a minimization of autofluorescencefrom the body surface to be measured. The probe will respond toflorescent illumination of a specific wavelength and will then emitlight at a different wavelength.

Other dyes include, without limitation, any of the FDA approved dyes touse in food, e.g. FD&C Blue No. 1 E133, FD&C Blue No. E132, FD&C GreenNo. 3, Orange B(3), FD&C Red No. 3 E127, FD&C Red No. 40(3) E129, FD&CYellow No. 5 E102, FD&C Yellow No. 6, D&C Black No. 2 &3, D&C Red No. 6,7, 17, 21, 22, 27, 28, 30, 31, 33, 34, 36, 40, D&C Violet No. 2, etc.

In alternative embodiments the biotag is imaged by one or moremodalities that may include, without limitation, optical coherencetomography, Raman spectroscopy, photo acoustic imaging, ultrasoundimaging, endoscopy, and the like.

Calibrated Intensity—The effective intensity or brightness as viewed bya medical imaging device of an object or area of interest, such as afiducial, can be known, either because the object has been manufacturedor created to have a known and documented intensity or because theintensity has been measure or compared to a known standard.

Cavity—A cavity in a camera body may be used to accept a battery,storage card, wireless interface, communications cable, remote viewingscreen, remote control accessory, mechanical mount or other accessory.The cavity may completely contain the item, as is common for batteriesand storage cards, or it may partially contain the accessory, such asmight be used for a wireless communication card with an antennaprojecting from the body of the camera, or the cavity may simply be arecessed connector for the component. Some cameras come from themanufacturer with sufficient internal storage memory that auser-provided external storage card is not necessary.

Tissue/cavity surface—A layer of tissue covering the body surface orinternal body cavities, such as the lining of the digestive tube, themouth, pharynx, the terminal part of the rectum, the lining cells of allthe glands which open into the digestive tube, including those of theliver and pancreas; the epithelium of the auditory tube and tympaniccavity; the trachea, bronchi, and air cells of the lungs; the urinarybladder and part of the urethra; and the follicle lining of the thyroidgland and thymus. In some instances, the surfaces come in contact withair, or fluids such as the skin, lung, colon, etc.

Diseased cell or tissue—A cell or tissue that is different or changedfrom the normal cell or tissue.

DSLR (digital single-lens reflex)—An SLR (single lens reflex camera)with an electronic image sensor.

Early stage disease—includes early stages of disease development priorto becoming recognizable or diagnosed using conventional methods. Anexample of early stage cancer is when a few cells are present beforeneoangiogenesis or vascularization, or a micro-foci. An example ofnon-cancer skin disease is the initial body response to a pathologicalsignal or an antigen.

Cavity/tissue surface lining lesion. As used herein, the term refers tocancerous and pre, cancerous lesions of a cavity/surface lining. Thesecould be ectodermal, endodermal or mesodermal tissues, particularlythose tissues lining body cavities or surfaces in which a lesion ispresent within about 2.5 cm of an accessible surface, and which can beimaged by the methods of the invention. These tissues include, but notlimited to the skin, the mucous membrane of the pharynx (including mouthand nose), the pharyngeal ducts, the larynx, the upper esophagus, thebronchial mucosa, the lining of the milk ducts, the small curvature ofthe stomach, the bile ducts of the liver, the gall bladder, the ducts ofthe pancreas, the urinary bladder, urethra and renal pelvis, the cervixuteri, and the lower part of the rectum.

The predominant cells of the ectodermis can be squamous epithelialcells, and certain cancers of interest can be squamous cell carcinomas(SCC), e.g. SCC of the lips, mouth, esophagus, urinary bladder,prostate, lung, vagina, and cervix. Other cancers of interest include,without limitation, basal cell carcinomas, melanomas, etc. For imaginglesions other than skin, e.g. bladder cancer, cervical cancer, etc., anendoscope may be preferable.

Effectively equivalent imaging sequence—This refers to taking aphotograph or image that is functionally equivalent to anotherphotograph or image taken under the discussed different condition. Forexample, a camera with an original internal infrared blocking filterperforms a certain way, particularly with regard to the way variousvisible colors are rendered and the performance of the autofocus withinthe camera. The same camera with the original internal infrared filterremoved and an external infrared filter added may then takesubstantially the same quality of photographs or images, includingsubstantially the same autofocus performance. In this example theperformance of the unmodified and modified camera would be effectivelyequivalent. The imaging sequence includes autofocus and auto-exposure,if the user has enabled these features. Functional equivalence in amedical context means the two comparative images can have the same orcomparable medical value, but are not necessary visually identical.

Excitation light—Excitation light source, spectral band, or filter topass excitation light must have some light overlapping with theexcitation band of the subject of interest, such as the fluorescentportion of a biotag. However, the critical feature of excitation lightor excitation light filters is that it has the lowest possible amount oflight in the emission band of the subject of interest, which is ourdefinition herein. Thus, the excitation light may not necessary havegood spectral alignment with the excitation band of the subject ofinterest.

Structured light—Illumination of the object with a known pattern. Forexample illumination with multiple distinctive lines create line patternon the object. Those lines can be used for 3D and roughness analysis ofthe object.

Exposure—Exposure is the process within a camera used to take a picture.The result of an exposure is one or digital images stored in theinternal memory within the camera. The storage may be temporary; forexample, the digital image data may be then transferred to a storagemodule, communicated via a communication port on the camera, ortransmitted wirelessly via a wireless communication port on the camera.

Fluorescent imaging range—optimal imaging range for animals and humansis from 650 nm-850 nm.

Fluorescent marker or label—an entity that is able to emit fluorescencelight that can be captured by a camera.

Industrial imaging system—This is an imaging system primarily designedfor specialized, non-consumer applications, such as research andmedical. The system is comprised of separate components, which may ormay not be co-located in a single container, and may or not beconsidered portable. Components such as optics/sensor, illumination,image processing, memory, power supply, processor, and user-interfacemay be separated. Often, some of the components are off-the-shelfcomponents, such as a processor, PC, or lens.

Integrated imaging system—This is a self-contained camera containing thefollowing components: case, power-supply, lens, image sensor, imagestorage memory, user controls, user display, internal controlelectronics including stored instructions for an embedded processor, &internal image processing logic including stored instructions for anembedded processor. A consumer or professional digital single-lensreflex (DSLR) camera is one example of an integrated imaging system. Theintegrated imaging system may have interchangeable lenses, although thisis not a requirement. The integrated imaging system may have anautofocus capability, such as a mirror-less contrast detection autofocusmethod or a phase detection method using a mirror and a separate sensor.The lens may have macro-focusing capability. The integrated imagingsystem may have removable image storage modules and/or have cable forcommunicating stored images, and/or a wireless communications port forcommunicating stored images. An integrated imaging system does notrequire connection to an external computer for operation, although suchconnection may be optional. An integrated imaging system is distinctfrom an industrial, medical or compound imaging system where requiredcomponents and/or functionality are split between two or more physicalenclosures and one of the enclosures is or contains a computer.

Internal image storage memory—This may be permanent image storage withinthe camera body or may be provided by a removable plug-in module in acavity within the camera body provided for this purpose.

IR, or infrared light—can include near infrared wavelengths.Approximately the band from 650 nanometers to 4,000 nanometers.

Light baffle—A lightproof wall, material or container, which blocksstray or ambient light from entering the optics of the camera,predominantly between the entrance to the optical system and thepatient. The baffle may be in the form of a truncated rectangular orconical pyramid. The baffle may consist entirely or in part of aflexible material, such as black cloth, and/or rigid material such asblack paper, plastic, metal or other opaque, non-reflective material.Part or all of a baffle may comprise a cloth-like covering over the topof both the camera and patient, extending downward and around thepatient and camera such that most or all of the ambient light is blockedfrom entering the optical system of the camera. In one embodiment thelight baffle is a rigid, hollow, pyramidal tube attached temporarily orpermanently at the narrow end to the camera with the wider end placed orpressed against the patient.

Linear distance reference on the patient's skin—This is a ruler, marksor other means, within the field of view, such that the whole or part ofa photograph or image of the patient's skin may be dimensionallymeasured in linear units.

Macro environment—The cells or tissue in the proximity surrounding adiseased cell or lesion. Typically the macroenvironment, as used herein,refers to the extravascular space in the region of a lesion, includingthe outer walls of the vasculature.

Macro lens—Traditionally this referred to lens that imaged an objectapproximately as large or larger at the image place as the actualobject. However, with the advent of modern high-density image sensors,we use the definition herein that a macro lens, macro focus, or macroimaging refers to the having a visible resulting image, when viewed atusable and appropriate resolution in either a hard copy or anelectronically presented image where the viewed image is at least aslarge as the original image. For example, if imaging a patient molewhose actual diameter is one millimeter, a macro image could be anyimage of that mole displayed with a visible diameter of at least aone-millimeter.

Measuring a lesion—A lesion, such as a mole or cancer, is often measuredfor diagnostic and medical record keeping purposes. Such a measurementmight be a diameter or circumference or thickness. Such measurement maybe manual or automatic.

Microneedles (MN), as used herein, refers to one or moremicro-projections (e.g., arranged in one or more rows, one or morecolumns, staggered rows and/or columns, or an array comprising aplurality of micro-projections), generally ranging from about 1 μm toabout 5 μm or about 25 μm to about 2000 μm in length, which are attachedto a base support. An array may comprise 10², 10³, 10⁴, 10⁵ or moremicroneedles, and may range in area from about 0.1 cm² to about 100 cm².Application of MN arrays to biological membranes creates transportpathways of micron dimensions, which readily permit transport ofmacromolecules such as large polypeptides. In some embodiments of theinvention, a microneedle array is formulated as a transdermal deliverypatch. MN arrays can alternatively be integrated within an applicatordevice which, upon activation, can deliver the MN array into the skinsurface, or the MN arrays can be applied to the skin and the device thenactivated to push the MN through the skin surface. MN can be used todeliver the biotag or the fiducial marking to the skin.

Various materials have been used for microneedles. In some embodiment,biodegradable materials into which the biotag can be incorporated are ofinterest. Such materials include various biodegradable or biocompatiblepolymers or cross-linked monomers, as known in the art. Thebiodegradable materials can be bioabsorbable. The biotags can beabsorbed or incorporated to a target region as the microneedlesbiodegrade. The dose of biotag or fiducial to be delivered will vary,and may range from at least about 1 ng/microneedle array, at least about10 ng, at least about 0.1 μg, at least about 1 μg, at least about 10 μgor more in a single array. MNs may be fabricated with a wide range ofdesigns (different sizes and shapes) and different types (solid, hollow,sharp, or flat), and may be in-plane and/or out-of-plane.

Polymeric MNs can provide biocompatibility, biodegradability, strength,toughness, and optical clarity. To accurately produce the micro-scaledimensions of polymer MNs, a variety of mould-based techniques, such ascasting, hot embossing, injection molding, and investment molding may beused, e.g. beveled-tip, chisel-tip, and tapered-conepolydimethylsiloxane (PDMS) molds. Polymeric materials of interest forfabrication include without limitation; poly (methylmetha-acrylate)(PMMA), poly-L-lactic acid (PLA), poly-glycolic acid (PGA), andpoly-lactic-co-glycolic acid (PLGA), cyclic-olefin copolymer, poly(vinyl pyrrolidone), and sodium carboxymethyl cellulose. Sugars havealso been used to fabricate the MNs, such as galactose, maltose,aliginate, chitosan, and dextrin. Materials may be cross-linked throughion exchange, photo-polymerization, and the like.

As an alternative to a biodegradable microneedle, a microneedle may beused which is a hollow needle having an exposed height of between about0 and 1 mm and a total length of between about 0.3 mm to about 2.5 mm,usually between 30 to 34 gauge. Usually, the microneedle is a hollowneedle having a length of less than about 2.5 mm. The biotags aredelivered into the skin to a depth of at least about 0.3 mm and no morethan about 2.5 mm by the microneedle. The biotags can be deliveredthrough the hollow portion of the microneedle. The biotags can be storedand/or delivered via a channel in the microneedle. In some alternativeembodiments, the microneedles can be coated with materials, such asbiotags.

Near IR—Approximately the band from 650 to 1400 nanometers. Herein, theterm “IR” or “infrared” generally refers to the near IR band or includesthe near IR band, unless stated otherwise.

Medical Professional—Ideally a physician such as a dermatologist.However this term applies to any physician, healthcare provider, othermedical personnel, or technician using this invention. The medicalprofessional can include any individual with training or knowledge ofuse of the systems and methods described herein. In some embodiments, itcan include the patient.

Removable optical filter—The filter may be completely removed or may berepositioned so that it is no longer in the optical path of the camera.The movement of the filter may be completely manual, or may be assistedby a powered mechanism whose operation is controlled by a user; or maybe completely automated. More than one filter may be involved. Forexample, one or more filters may be on a slide, where one filter isselected by moving the slide. One or more filters may be in a rotatingcarousel. One or more filters may rotate, or flip on a hinge out of theoptical path. A suitable hinge design is similar to the design onpopular flip-up sunglasses.

SLR—Single lens reflex camera.

Transmit or block wavelengths of light—An ideal filter may becharacterized by passing 100% of light within a pass band and passing 0%of light outside that pass band. Such an idealized filter has anassociated spectral curve in the shape of a rectangle with at least onevertical edge. However, available filters, as one trained in the artappreciates, have sloped sides in their spectral curve. In addition, thelight passed in the pass band is often slightly less than 100% and theamount of light passed outside the pass band is often more than 0%. Thismeans that there is a range of wavelengths of light in which the amountof light passed by the filter varies, perhaps monotically or perhapsnon-monotically, from within the pass band to outside the pass band.Thus, there is no exact cutoff frequency defining at least one side ofthe pass band. Filters may also be low pass or high pass. By convention,depending on the type of filter and the application, the stated passband threshold might be at the wavelength where 50% of the light passesthrough the filter, or might be determined by some other metric. When werefer herein to a spectra, pass band, range, excitation band, emissionband, transmission or blockage of light, or other reference to a rangeof light wavelengths, we are using the accepted terms of the art todescribe the band including the understanding that passing and blockinglight may be less than 100% or more than 0% respectively.

Visible light—Approximately in the band from 400 to 700 nanometers.Within the light band from 650 to 1400 nanometers—The ideal band forfocus is at the same infrared wavelength as the peak emission wavelengthof a fluorescent emission from the biotag on the patient. However, thereis typically considerable latitude in exact range of wavelengths usablefor autofocus. The autofocus does not necessary have to focus on allwavelengths or any wavelengths from 650 to 1400 nm, but rather has tofocus on the emission wavelengths for the biotags in use. In oneembodiment using Cy5.5 as the fluorescent compound this range isapproximately 690 to 750 nm. In another embodiment using ICG as thefluorescent compound this range is approximately 815 to 915 nm.

Additional aspects and advantages of the present disclosure will becomereadily apparent to those skilled in this art from the followingdetailed description, wherein only illustrative embodiments of thepresent disclosure are shown and described. As will be realized, thepresent disclosure is capable of other and different embodiments, andits several details are capable of modifications in various obviousrespects, all without departing from the disclosure. Accordingly, thedrawings and description are to be regarded as illustrative in nature,and not as restrictive.

BRIEF DESCRIPTION OF THE DRAWINGS

The novel features of the claimed invention are set forth withparticularity in the appended claims. A better understanding of thefeatures and advantages will be obtained by reference to the followingdetailed description that sets forth illustrative embodiments, and theaccompanying drawings or figures (also “FIG.” or “FIGs.” herein) ofwhich:

FIG. 1a shows a block diagram of a camera.

FIG. 1b shows a wire-frame isometric view of the camera, in oneembodiment.

FIG. 2a shows one embodiment of a method of diagnosis.

FIG. 2b shows one embodiment of a method for image transfer and moleclassification.

FIG. 3a shows a cutaway view of one embodiment of the camera.

FIG. 3b shows an isometric view of the camera from the back.

FIG. 4 shows a black and white photographic image from the inventionshowing a mole and two fiducials using white light.

FIG. 5 shows two black and white photographs of a skin tumor and benignskin growth and fiducials in IR light and in white light.

FIGS. 6a and 6b show one embodiment of a fiducial and a variation,respectively.

FIG. 7 shows one embodiment of fluorescent marker selection andassociated light spectra and filter spectra.

FIGS. 8a and 8b show a benign mole topically treated with a biotag invisible light and IR light, respectively.

FIGS. 9a and 9b show a recurring melanoma mole topically treated with abiotag in visible light and IR light, respectively.

FIG. 10 shows an X-Y graph of two features identified automatically fromimage analysis of sample images. Three types of moles are shown asdifferent symbols on the graph.

FIG. 11 shows a flowchart of image processing.

FIG. 12 shows an image of mole overlaid with two lines of patternillumination.

FIG. 13 shows an exemplary curve for a single, hybrid filter.

DETAILED DESCRIPTION

While various embodiments of the invention have been shown and describedherein, it will be obvious to those skilled in the art that suchembodiments are provided by way of example only. Numerous variations,changes, and substitutions may occur to those skilled in the art withoutdeparting from the invention. It should be understood that variousalternatives to the embodiments of the invention described herein may beemployed.

In one aspect, systems, compositions and methods are provided forimaging of cavity and/or tissue lesions. Various aspects describedherein can be applied to any of the particular applications set forthbelow, alone or in combination, or for any other types of imagingsystems. The embodiments described herein may be applied as a standalonesystem or method, or as part of an integrated medical diagnostic and/ortreatment system. It shall be understood that different aspects can beappreciated individually, collectively, or in combination with eachother.

Methods of Analysis

Systems and methods may be provided to image and/or analyze a targetregion. In some embodiments, the target region may include acavity/tissue surface. The cavity/tissue surface that is to be analyzedcan be identified, e.g. by the presence of a suspected lesion. In someembodiments a target area may be the surface of cavity/tissuecompartments where there is a suspicion of cancer cancerous orpre-cancerous lesion, which may be referred to as an area of interest ordiagnostic area of interest. Surfaces include skin, cervix, oral mucosalsurfaces, bladder, and the like. In some embodiments the surface isskin.

A suspected lesion can be less than about 0.5, 1, 2, 3, 4, 5, 6, 7, 8,9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, or 20 mm in diameter (or anyother dimension such as radius, length, width, height, perimeter, orcircumference). The suspected lesion may be above 20 mm in diameter. Asuspicious lesion may be asymmetric or symmetric. A suspicious lesionmay have regular or irregular borders. The lesion may or may not containexcess pigment or melanin. The lesion may or may not contain more than 1color. The lesion may or may not be evolving. The lesion may or may notinduce a noticeable sensation to the patient. The cavity/tissue surfacemay be cleaned with water, alcohol, and/or a surfactant prior to theassay, or by other means as typical in a medical professional'spractice.

The cavity/tissue surface is optionally preconditioned to increasedelivery of the biotag through the surface. For preconditioning, apenetration enhancer can be applied to the cavity/tissue surface priorto contacting the surface with the biotag. Penetration enhancers caninclude sulphoxides (such as dimethylsulphoxide, DMSO), azones (e.g.laurocapram), pyrrolidones (for example 2-pyrrolidone, 2P), alcohols andalkanols (ethanol, or decanol), glycols (for example propylene glycol,PG, a common excipient in topically applied dosage forms), surfactants(also common in dosage forms) and/or terpenes. DMSO is of particularinterest. The concentration of penetration enhancer may range from10-90% or 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80,85, 90% or 10-15, 15-20, 20-25, 25-30, 30-35, 35-40, 40-45, 45-50,50-55, 55-60, 60-65, 65-70, 70-75, 75-80, 80-85, or 85-90% or 10-20,20-30, 30-40, 40-50, 50-60, 60-70, 70-80, 80-90%. In some instances, ifa penetration enhancer is DMSO, a preferable range of DMSO concentrationmay be between 40-70%.

Optionally, as an additional preconditioning step, or in combinationwith preconditioning using a penetration enhancer, a blocker can beadded to the vehicle. The blocker may be a protein not associated withthe lesion of interest, e.g. albumin, casein, etc. The blockerconcentration may range from 0.01 to 10%, or 0.01-0.1, 0.1-0.2, 0.2-0.3,0.3-0.4, 0.4-0.5, 0.5-0.6, 0.6-0.7, 0.7-0.8, 0.8-0.9, 0.9-1, 0.1-0.2,0.2-0.3, 0.3-0.4, 0.4-0.5, 0.5-0.6, 0.6-0.7, 0.7-0.8, 0.8-0.9, 0.9-1,1-2, 2-3, 3-4, 4-5, 5-6, 6-7, 7-8, 8-9, or 9-10%. A preferable blockerconcentration can be between 0.2%-2%.

In some embodiments the invention includes a method of enhancing thetransfer of an agent across intact skin, the method comprisingpreconditioning the skin by topically applying an effective dose of apenetration enhancer in the absence of the agent, for a period of time(e.g., from 5 to 30 minutes); and topically applying the agent in avehicle comprising a penetration enhancer, wherein transfer of the agentacross intact skin is increased relative to transfer in the absence ofpreconditioning.

In some embodiments, the biotag is next applied to the cavity/tissuesurface. The biotag is generally formulated in a physiologicallyacceptable vehicle, which optionally comprises a penetration enhancer asdescribed above. The biotag can be applied topically to the region ofinterest, or by subdermal injection with a microneedle to the area ofinterest or diagnostic area of interest. In some embodiments,penetration of the biotag is within about 2 cm of the surface. Thebiotag may penetrate about or less than 0.1 cm, 0.3 cm, 0.5 cm, 0.7 cm,1.0 cm, 1.3 cm, 1.5 cm, or 2.0 cm. Where administration is by subdermalinjection it will not be necessary to include a penetration enhancer inthe formulation. In the methods of the invention the biotag is notinjected into the bloodstream. For example this approach being lessinvasive is also less subject to side effects and does not require asterile needle. Topical application provides a number of benefits, inbeing non-invasive, not requiring a sterile needle, and it is alsoeasier for the medical professional. Methods of application includes theuse of micro-needles, nano-needles, active patches and passive patches.Topical application includes the use of a gel, such as a gel that needsto be activated, either chemically or mechanically, from a storage stateto a usable state.

The biotag formulation can comprise a solvent, and optionally blocker,skin penetrator and/or an enhancer, ion-pairing agent, co-solvent and/orhumectants and/or thickeners, alone or in various combinations. Thesolvent functions as the carrier for the biotag. The skin penetratorfacilitates transdermal penetration. The enhancer reduces the backgroundnoise by inducing efficient stratum cornea transfer. The blocker blocksexposed epitopes in the skin and prevents or reduces non-specificbinding of the biotag to these epitopes. The formulation may be a liquidor gel, e.g. a thickener may be included to generate a gel-likeformulation or in a formulation composed of micelles or reverse micellesin a liquid or spray dispenser. With a liquid formulation, a barrier isadded in some embodiments to prevent the liquid from rolling off theskin. This barrier can be a gel-like substance that generates a surfacetension for an appropriate quantity of the transdermal penetrationcombination, or a mechanical barrier, such as a polymer.

Alternatively the biotag can be adhered to a membrane and dried, where asolvent, including for example a penetration enhancer, is used to wetthe membrane immediately prior to contact with the cavity/tissuesurface.

Desirably the formulation provides for a rapid release of the biotagagent from the vehicle to the cavity/tissue surface; the biotag could berapidly transported across the cavity/tissue surface to produce a lowbackground image; residual vehicle components preferably should notdissociate from the biotag after transport, so not to interfere withbiotag binding; be non-toxic or sensitizing; be acceptable to FDA andEMA regulatory reviewers; optionally contain a viscosity building agentso the formulation stays in place until the vehicle penetrates thesurface; and/or be easy to remove the residue from the surface. A rapidtransport may be less than about 5, 10, or 15 minutes.

Solvents or cosolvents include water, saline, DMSO, ethanol, proplyeneglycol, PEG 300, N-methyl pyrollidone, isopropyl myrstate, labrafil,labrasol, gelucires, surfactants, dodecyl pyridinium chloride,poloxamer, sorbitol, oils, glycerin, azone; diethylene glycol monoethylether; nonoxynol-9; NMP; cyclodextrins; surfactants (such as tween 80and cremophor); vitamin E TPGS; and the like as known in the art.

Ion pairing agents include ethanolamine, triethanolamine and dodecylpyridinium chloride; oleic acid and sodium lauryl sulfate; and manyothers.

Co-solvent and humectants include propylene glycol or isopropylmyrstate.

Thickeners include hydroxyethyl cellulose, carbomer or starch.

The formulation may be provided as a lyophilized substance in single ormultiple use units. It may be reconstituted by a pharmacist or themedical professional before use. Alternatively it is provided in astable formulation where no reconstitution is required and may be useddirectly by the medical provider.

The dose of the biotag may be 1 fg-1 g, 1 fg-1 pg, 1 pg-1 ng 1 pg-1microg, 1 microg-1 mg, 1 mg-1 g, 1-10, 10-20, 20-30, 30-40, 40-50,50-60, 60-70, 70-80, 80-90, 90-100, 100-150, 150-200, 200-250, 250-300,300-350, 350-400, 400-450, 450, 500, 500-550, 550-600, 600-650, 650-700,700-750, 750-800-800-850, 850-900, 900-1000 fg, 1-10, 10-20, 20-30,30-40, 40-50, 50-60, 60-70, 70-80, 80-90, 90-100 pg, 1-10, 10-20, 20-30,30-40, 40-50, 50-60, 60-70, 70-80, 80-90, 90-100, 100-150, 150-200,200-250, 250-300, 300-350, 350-400, 400-450, 450, 500, 500-550, 550-600,600-650, 650-700, 700-750, 750-800-800-850, 850-900, 900-1000 ng, 1-10,10-20, 20-30, 30-40, 40-50, 50-60, 60-70, 70-80, 80-90, 90-100, 100-150,150-200, 200-250, 250-300, 300-350, 350-400, 400-450, 450, 500, 500-550,550-600, 600-650, 650-700, 700-750, 750-800-800-850, 850-900, 900-1000microg, 1-10, 10-20, 20-30, 30-40, 40-50, 50-60, 60-70, 70-80, 80-90,90-100, 100-150, 150-200, 200-250, 250-300, 300-350, 350-400, 400-450,450, 500, 500-550, 550-600, 600-650, 650-700, 700-750, 750-800-800-850,850-900, 900-1000 mg, 1-10, 10-20, 20-30, 30-40, 40-50, 50-60, 60-70,70-80, 80-90, 90-100, 100-150, 150-200, 200-250, 250-300, 300-350,350-400, 400-450, 450, 500, 500-550, 550-600, 600-650, 650-700, 700-750,750-800-800-850, 850-900, 900-1000 g. The preferred amount biotag in oneembodiment is between 1 fg-0.1 microg. The units may be read so that fgis femtograms; pg is picograms; ng is nanograms; microg is micrograms;mg is milligrams; g is grams.

A preferable volume of biotag applied to the cavity/tissue surface isbetween 50 to 150 microliter per square centimeter. Depending on theapplication and the embodiment, the biotag can be applied in a volume of50, 100, 150, 200, 250, 300, 350, 400, 450, 500, 550, 600, 650, 700,750, 800, 850, 900, 950, or 1000 microliters.

Depending on the application and in one embodiment, the biotagformulation may be 10-90% or 10, 15, 20, 25, 30, 35, 40, 45, 50, 55, 60,65, 70, 75, 80, 85, 90% or 10-15, 15-20, 20-25, 25-30, 30-35, 35-40,40-45, 45-50, 50-55, 55-60, 60-65, 65-70, 70-75, 75-80, 80-85, 85-90% or10-20, 20-30, 30-40, 40-50, 50-60, 60-70, 70-80, or 80-90% DMSO.

The biotag interacts with the tissue and binds to the appropriatebinding partners, a process that typically takes several minutes. Theexcess, unbound biotag material is then removed. In some examplesremoval may occur via washing or wiping with water or saline solution,with or without a detergent. Depending on the application and theembodiment, excess (non-bound or non-retained) biotag can be removedafter 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 18,20 minutes or within 20-25, 25-30, 30-35, 35-40, 40-45, 50-55, 55-60minutes, or within 1-2, 2-3, 3-4, 4-5, 5-6, 6-7, 7-8, 8-9, 9-10, 10-11,11-12, 12-13, 13-14, 14-15, 15-16, 16-17, 17-18, 18-19, 19-20, 20-21,21-22, 22-23, 23-24 hours or within 1-2 days. A preferable time ofbiotag application is between 2 to 15 minutes and less than 2 hours.Retention of the biotag in the cavity/tissue compartment occurs when theappropriate binding partner is found in the lesion macroenvironment.

In one embodiment, prior to imaging the area of interest, calibrationmarkers in the form of fiducials can be applied proximal to the lesionin the area of interest. Fiducials are placed on the patient or fixed tothe imaging device. The fiducials can be removably provided on thepatient, drawn on the patient, affixed (removably or permanently) to theimaging device or provided separately from the imaging device. Dependingon the application, images may be acquired prior to application of thebiotag as well as after application. Images may be acquired using acamera, or any of the devices, systems and methods described within thisspecification.

In some embodiments, a camera takes two images of the area of interest.One image (color or gray scale) can use visible light and the secondimage can use light in the emission spectra of the biotag. The emissionlight may be activated by light from the camera in the activation bandof the biotag.

Images are typically transferred out of the camera for further medicalanalysis. Such analysis can include comparing statistical featurescalculated on both image, merged or overlaid image composed of both thevisible light image and the emission light image of the area ofinterest. In an alternative embodiment, the pair of images is presentedas a pair, although the former presentation is preferred. Suchstatistical features can include dimensions, brightness, intensity,contrast, color, mapped 3D features or texture, or any other featuresdiscernible from images.

The images are analyzed to identify the intensity of the reporter tagretention of the imaging agent in the surrounding tissue and the patternof its retention in the tissue of interest. The image intensity iscalibrated to the intensity of the calibration tags.

In some embodiments the calibration tag contains a unique barcode orother identifier for identification of the lesion imaged. (Barcodegenerally refers to information which is unique for a specific tag,e.g.: linear barcode, 2D metric barcode) The calibration tag can includea visual identifier.

The analysis output can be stratified into the classificationsreflecting the probability of the lesion being a tumor. FIGS. 2a and 2bprovide examples of methods of analysis.

Tumor Detection

In some embodiments, diagnostic methods are based on imaging of anexternally applied biotag that specifically interacts with acancer-associated entity of interest, and thus which distinguishesbetween pathological and non-pathological lesions on a surface of thebody. Markers of interest include markers expressed on neoplastic cells,markers selectively expressed on neoplastic cells or their surroundingof the microenvironment, makers associated with tissue remodeling,markers on immune cells recruited to the skin under investigationfollowing the application of an entity that may recruit the response,markers expressed on cells associated with tumor angiogenesis; markerssecreted by neoplastic cells; and the like, particularly cell surface orsecreted markers. Optionally the marker is compared to a negative and/orpositive control, e.g. a fluorophore in the absence of a binding probeas a negative control; and the like. Alternatively, optionalinstructions depicting positive and negative images may be included inkits of the invention.

Topical Application

In some embodiments of the invention, an optimized or improved procedurefor image acquisition with minimal or reduced background noise utilizesthe following process for biotag application. The tissue cavity surfacecan be first cleaned with a cleaning solution, comprising typicallyalcohol, or surfactant, or saline or water or some combination of these.The lesion is contained using a barrier, or alternatively the biotag isapplied in a gel formulation. Examples of such barriers includepetroleum jelly, a polymer applied directly to the skin, or anotherbarrier means. Next, the stratum cornea is prepared. A preconditioningformulation of penetration enhancer with or without blocker is topicallyapplied. After a short time for incubation of the preconditioningformulation, the biotag formulation is applied to the cavity/tissuesurface. After a time for incubation of the biotag formulation, theexcess can be washed away using saline or water with or without adetergent or other surfactant. The time for incubation can be apredetermined amount of time, such as time quantities describedelsewhere herein. The time for incubation can be flexible and dependenton one or more indicators.

Application may alternatively be via intradermal or subdermal injection,instead of topical. Application may alternatively be by spray. Methodsof application also include the use of micro-needles, nano-needles,active patches and passive patches

Camera

An imaging device may be used to image a target area of interest. Theimaging device may be a camera having one or more components,characteristics, or features described herein. FIG. 1a , FIG. 1b , FIG.3a , and FIG. 3b provide examples of imaging devices that may be used inaccordance with embodiments of the invention. Preferably, the cameraimaging the area of interest has autofocus and is able to focus on thelesion itself. Such a system might be theoretically possible if thelesion emits fluorescently from the biotag. However, if there are nocancerous cells in the lesion then the biotag will be missing and therewill be no light source on which the camera can focus. Therefore, tohandle the case of non-cancerous lesions, this invention uses theaddition of a novel fiducial. The fiducial comprises a fluorescentmarker or tag which comprises either the same fluorescent compound asthe fluorescent marker present on the biotag, or comprises a compoundthat emits light in a compatible spectra as the biotag (for example FD&CGreen No. 3) so that it can be detected by the camera optics and used asa target for autofocus. Compatible spectra include, for example, aspectrum that comprises excitation light in the spectra of theexcitation of the biotag, and light emission of the fiducial comprises aspectral emission within the spectrum of the biotag emission. In somecases, common food coloring is be used as the fluorescent compound inthe fiducial. The fiducial can be applied directly to the tissue/cavitysurface or on a medium that is then applied to the surface, for examplea sticker or transferred from a medium to the skin, for example atemporary or permanent tattoo.

This method of this invention can include both the use of autofocus inthe camera and the use of a fluorescent fiducial. It can also includethe use of autofocus independently, the use of a fluorescent fiducialindependently, or neither of these features.

In a preferable embodiment, a user input on the integrated imagingdevice changes the autofocus from visible light to infrared light. Thisis required or preferable when the autofocus is based on phase detectiondue to the different width of the phase lines when properly focused inIR as compared to visible light. For contrast-based autofocus, no changein the algorithm or constants is needed and thus no user input on thecamera is needed. Alternatively, the means to select visible or IRautofocus is determined automatically from which illuminator is on,respectively. One means, the preferred embodiment, uses the mode dial orpush button on the camera to for this selection. Either a “custom mode”provided on the dial is programmed for this purpose, or one of the othermodes, such as “portrait,” or “night” mode is taken over for thispurpose. Touch-screen based camera control systems are ideallyextensible to provide specifically for this selection explicitly. Anintegrated imaging device can permit an autofocus change betweendifferent spectra of electromagnetic radiation, such as visible lightand infrared light. Such autofocus can occur manually with user input,or automatically without requiring user input. Such autofocus may occurwith aid of a processor.

In some embodiments fiducials are not used, or they are not suitable forautofocus. This may be remedied by adding an additional light sourcethat emits light in comparable spectra to the emission wavelengths ofthe biotag, then using the auto-focus to focus on the subject using thislight. In one embodiment, this light source is provided integral to theinvention, using narrow-band LEDs or LEDs with a spectral filter. Afterthe auto-focus completes, this light source is turned off andimmediately the excitation light source is turned on, and the picture istaken. Any light source known in the art may be utilized, which mayinclude light emitting diodes (LEDs), electron stimulated light sources,incandescent light sources, electroluminescent light sources, gasdischarge lamps, or high-intensity gas discharge lamps. Light sourcesmay be electrically powered and/or may utilize chemical or biologicalluminescence.

There is nothing in this invention that precludes the use of industrialcameras or other imaging devices or technologies. As an example, anintegrated imaging system permits the addition of user-providedsoftware. A first example is a camera running the Android OS with a USBinterface. A user-provide app, running on the camera, performs themethods described herein; while the USB interface provides an interfaceto functionality not provided originally in the camera, such as turningon and off illumination, moving filters, and the like. The camera mayoptionally include a local memory and/or processor. The local memory maystore non-transitory computer readable media comprising code, logic,instructions to perform one or more steps. The processor may be capableof performing one or more steps, optionally in accordance with thenon-transitory computer readable media. A second example of analternative imaging system comprises off-the-shelf optics and an imager,with a single-board computer providing a processor and memory, or amemory interface, for implementing the methods described herein. As athird example, a portable electronic device such as a tablet or smartphone provides the platform for an app, memory, and the user interfaceof this invention. The portable electronic devices uses a built-ininterface such as USB or Bluetooth to interface to requiredfunctionality not initially included in the portable electronic device.The camera may include one or more of the functionalities on-board ormay communicate with one or more external devices that provide one ormore of the functionalities described herein. The camera may communicatewith an external device via a wired or wireless communication. Thecamera may communicate directly with an external device or devices, ormay communicate with the external device or devices over a network, suchas a local area network (LAN), wide area network (WAN) such as theInternet, telecommunications network, or any other network. Such camerasmay also find use in the methods of the invention as an endoscope, i.e.a general industrial camera with fiber optics to transfer the image, asis known in the art.

Operation of the camera includes various degrees of manual operation andautomatic operation, depending on embodiment. In a more manualembodiment, the two photographs in IR light and visible light are takenseparately. The filters in the optical path are moved manually betweenexposures. The shutter release button is pressed once for each image tobe acquired. In a more automatic embodiment, “one button” operationtakes both images, automatically changing the filters and camera modesbetween the two exposures.

This second, “one button” embodiment can be implemented within thefirmware of the integrated imaging device, which is updated for thispurpose from the firmware provide by the manufacturer of the integratedimaging device. Alternatively, a separate controller can be used, whichis integrated into the camera of this invention, but is not internal tothe integrated imaging device. In the latter case, a microprocessor andcontrol logic comprise a typical implementation. Ideally, the “onebutton” is the existing shutter release on the integrated imagingdevice. However, it may also be a separate button, which is an input tothe separate controller. To move the filters, a simple motor can be usedwith a slide or hinges. In further alternative embodiments, the separatecontroller can be external to the camera.

Imaging

In accordance with an embodiment, the imaging process may take both avisible light (e.g., white light) color image and also an image usinglight in the emission spectra. These two images are taken in eitherorder. The emission spectra image typically uses as light sources onlythe emitted light from the detectable label components of the biotagand/or the fiducials. These light sources are activated by light in theactivation spectra of the fluorescent components, where the activationlight comes from the camera. However, an emission spectra image mayalternatively be acquired of light emitted in the range of a fluorescentlabel.

Generally, when the two images are analyzed together for a medicalpurpose, the visible light image shows what a person sees, such as amole, and the emission spectra image, because of the biotag and theother elements of this invention, shows the cancerous cells.

Generally when triple images are analyzed the structure illuminationimage is used to analyze the roughness of the mole and to segment thehair that obscures the mole. Hair might be filtered or subtracted fromthe image based on the hair segmentation.

An image of the area of interest in the emission spectra is to be takenprior to the application of the biotag for background subtractionproposes, in some embodiments.

In accordance with embodiments of the invention, multiple images may becaptured. The multiple images may be captured under different lightsource spectra or wavelengths. Multi-wavelength images may be captured.For example one or more white light source or fluorescence light sourcemay be used. One or more images may include analysis of features shownin the images. The images may be captured from the same angle or varyingangles. One, two, three, four or more images may be captured. The imagesmay be compared, contrasted, and/or overlaid.

Image processing and analysis may be manual or automatic, depending onembodiment. Maximizing the processing performed automatically is thepreferred embodiment. Computerized image processing may be performed inthe camera, using its embedded processor, or on a computer, tablet,smart phone or other electronic computational device. The steps of imageprocessing can be split among multiple devices.

The embodiment of using white light is not a requirement for thisinvention. The dual wavelength images have substantial diagnosticadvantage. However, for the simplest and lowest cost implementation,such as might be used for home use, or in remote clinics only singlewavelength range images, such as the fluorescent image is used. Forexample, seeing and identifying the mole border is generally practicalwith only the fluorescent image. Medical diagnoses may be incomplete insome embodiments, but any visible biotag fluorescence in the image is astrong indication that additional medical diagnosis and treatment isnecessary.

Automated Analysis

Steps of automated analysis include one or more of the following.

First, the lesion is automatically or manually outlined in the whitelight image. The fiducials can be identified in this image. The whitelight image is then overlaid on the fluorescent image using the elementsof the fiducial for this purpose. The lesion circumference is identifiedin the fluorescent image. This circumference is to be measured, in oneembodiment, using the measurement elements provided by the fiducial. Themeasurement element provided by the fiducial may be used for measurementcalibration, and measurement may occurred automatically with aid of aprocessor. Fluorescent intensity is compared in the fluorescent image inthe mole and around the lesion. The intensity is calculated compared tothe fluorescence in the calibrated portion in the fiducials. Theintensity can be calculated with aid of a processor.

Fluorescence is also calculated in the skin around the lesion and in thelesion and compared to fluorescence image taken before the biotag isapplied, if an image of the area of interest was taken prior toapplication of the biotag.

Additional features are extracted using image analysis algorithms toidentify features that distinguish and stratify moles according to levelof increasing malignancy. A processor may perform one or more steps orcalculations dictated by the analysis algorithms.

FIG. 4 provides an example of an image useful for or generated duringautomated analysis.

Analysis Presentation

Depending on embodiment, results of the analysis may be presentedgraphically in 2-D or 3-D figure. Results may be presented in black &white or color. The location of the lesion analyzed can be placed andlocated on graph. A database including a collection of lesion analyzedmay be included in the representation. Images of lesion in the databasemost similar to the patient's lesion may be selected from the databaseand presented. The database may be searched with aid of a processor forthe most similar images.

Depending on embodiment, a score may also be calculated to represent thelikelihood of a mole having a specific characteristics analyzed by thesoftware or a combined score of likelihood of a mole being melanoma or arecommendation for a biopsy or a recommendation for additionalevaluation. The score may be a numerical score along a scale that mayprovide likelihood of the detection of cancerous tissue. The score maybe used to recommend one or more medical action, such as biopsy oradditional evaluation. Additional factors, such as specific imagecharacteristics (e.g., dimensions, brightness, contrast, intensity,texture, color) can be used to provide qualitative evaluations orrecommendations for medical actions.

Depending on embodiment, measurements, metrics and scores may bepresented numerically or graphically.

Depending on embodiment, the visible light image and the emissionspectra image may be presented interactively by the use of an operableslider that shows 100% of the visible light image at one end and 100% ofthe emission spectra image at the other end, with variable portions ofeach image overlaid for intermediate slider positions.

Depending on embodiment, the emission spectra image may be presented ina contrasting color overlaid with the visible light image. For example,fluorescence may show as bright green.

3-D

In another embodiment of this invention, 3-D information about the areaof interest may be captured and/or analyzed. On method of 3-D imagecapture uses a structured light source, such as a set of parallel lines,which may be generated by a laser or diode. FIG. 12 provides an exampleof an image useful for 3-D analysis. A second method of 3-Dimage-capture uses two lenses and two image sensors offset in atraditional “3D camera” arrangement.

Depending on embodiment, 3-D image capture provides three importantmedical benefits. First, the surface of the lesion may be analyzed todetermine the quantitative elevation of the mole (if any) above thenormal skin surface. This helps in the determination of lesion type.Second, the surface of the lesion may be analyzed to determine theamount and quality of mole texture or roughness. This helps in thedetermination of lesion type. Third, hair may be identified by either ahuman or by an automated algorithm far easier and more accurately in a3-D image than in a 2-D image. Consistent and accurate identification ofhair is necessary or beneficial for automated hair removal. Removing ofhair from an image is important to improve the performance of otherautomated steps, such as determining the outline of a lesion.

Another problem with hair is that it can cause autofocus to focus on thehair, rather than the surface of the skin. Hair in an image mayinterfere with an automated algorithm to find the border of a lesion.Shaving a patient's skin can damage the skin or the lesion by causingmicro lesions on the skin surface. Excluding patients with hair for usein studies may bias the study. Thus automated hair removal permitsstudies with less possible bias.

Fiducials

The fiducial, in one embodiment, is placed on the patient's skin next tothe lesion of interest. FIG. 6a provides an example of a fiducialprovided in accordance with an embodiment of the invention.

The fiducial, in another embodiment, can be tattooed on the patient'sskin next to the lesion of interest.

A novel feature of one embodiment of this invention is the use of one ormore multi-function fiducials. This embodiment provides time savings,cost savings, reduces medical errors, and/or permits significantpost-photo automatic image processing and medical record keeping. Listedbelow are exemplary functions of fiducials, which are discussed infurther detail below. Note that this invention includes all or aplurality of combinations of these functions in, methods and uses of oneor more fiducials. In general, the more functions the better. Note,however, these individual functions or features are not isolated,independent benefits, but rather provide additional benefits when usedas group, these benefits more than the sum of the individual benefits ofthe features. The use of singular fiducial or plural fiducialsterminology is generally equivalent herein, unless specifically statedotherwise. Fiducial may refer to a single mark, a portion of a mark, ora set of marks, which may be on a single substrate for application ormay be on multiple substrates.

Table 2, below lists fiducial features numbers, which are then discussedindividual following the Table. Fiducial features appear alone or incombination in various embodiments.

TABLE 2 Feature No. Feature Description 1 Overall brightness of allvisible fiducials comparable to brightness of the disease area ofinterest for image exposure control in the emission band 2 Brightnessarea of calibrated intensity for use in determining a metric ofintensity in the disease area of interest 3 High contrast area for useas a focus or auto-focus target 4 Orientation of disease area ofinterest relative to the patient; anatomical terms of location on thepatient 5 At least one pair of locations on the fiducial of a knownlinear distance separation for use in determining a size metric of oneor more elements within the area of interest 6 At least one pair oflocations on the fiducial for alignment of multiple images of the samearea of interest taken at different light wavelengths 7 Identificationof a specific area of interest on a patient where multiple areas ofinterest are imaged on a single patient. For example, this might be anumerical mole id number. 8 A tracking identifier to uniquely identifythe specific medical diagnostic procedure being performed using thisfiducial 9 Manufacturer and lot number of fiducial, with optionalcalibration information 10 Area in which machine printed information maybe added at the time of the procedure 11 Area in which hand-writteninformation may be added at the time of the procedure 12 Pre-printedfiducials 13 Fiducials with a combination of fluorescent marks withsubstantially the same excitation and emission spectra as the biotag andvisible marks visible under visible light 14 Information to interfacewith an electronic medical records system.

Fiducial feature 1 serves provides appropriate fluorescent brightness inthe emission band to enable proper exposure, either preferably automaticexposure or manual exposure setting. Special area of the fiducial may beused to assure this, although generally the overall brightness of theentire area of interest is used for automatic exposure setting.

Fiducial feature 2 provides calibrated reference brightness so that thequantity or intensity of the biotag may be compared manually orautomatically to a known reference for medical diagnostic purposes. Suchcalibration may be integral to the manufacturing of the fiducial or maybe computed following the manufacture of the fiducial. The calibrationdata may be in reference to a specific lot number, and/or may be markedon the fiducial itself.

Fiducial feature 3 provides the ability to manually or preferableauto-focus the camera on the area of interest.

Fiducial feature 4 provides an important ability to locate theorientation of the area of interest with the anatomical orientation ofthe patient. As one example, an arrow on the fiducial may be alignedduring the procedure to point towards the proximal or posterior locationof the patient, as appropriate for the specific location and thepreference of the medical practitioner. Or it may help the medicalprofessional locate the mole in question where there may be multiplemoles in close proximity to one and another.

Anatomical terms of location include, for example: anterior, posterior,dorsal, ventral, left, right, medial, proximal, distal, etc.Additionally, a body part may be identified such as an arm, the back,etc.

Fiducial feature 5 provides a known linear distance in or next to thearea of interest to use in measuring any feature in the image, such asthe diameter or circumference of a mole.

Fiducial feature 6 provides an important component of this invention,which is the ability to align multiple images taken with differentwavelengths of light. Such alignment may be manual or preferablyautomatic. As discussed elsewhere herein, this feature allows a medicalprofessional to accurately compare the image seen with visible lightwith the image created by the biotag. The marks to implement thisfiducial feature must be visible in both visible light and in theemission band of the biotag. The marks do not have to appear identicalin both wavelength images, but they do have to clearly align.

Note that because the cameras may be hand-held, or because the patientmay move between exposures, images taken with visible and emissionspectra light may not be naturally aligned. Thus, the fiducial feature 6provides beneficial capabilities, as part of this invention, in oneembodiment.

Fiducial feature 7 provides the ability to identify multiple areas on apatient. A patient might have 20 similar looking moles on his back, forexample. It is important to know which mole is which when analyzing theresulting images.

Fiducial feature 8 permits an optional diagnostic procedure trackingcode to be present on the fiducial. This could be a pre-printed number,unique to each manufactured fiducial. Or, it could be an identifieradded at the time of the procedure. It might be human readable, machinereadable, or both. A machine-readable diagnostic procedureidentification aids substantially in permitting automated medical recordkeeping and in reducing medical errors. The tracking code may or may notbe visibly discernable. A signal may be emitted from a fiducial.

Fiducial feature 9 permits accurate tracking of fiducial manufacturingand quality. Like drugs, it is often valuable to identify a manufacturerand lot number for quality control, inventory, expiration date, andother purposes.

Fiducial feature 10 permits a manufactured fiducial with most of thesefeatures to be customized at the time of the diagnostic procedure. Suchcustomization would typically include the patient's name or patient IDnumber or other ID number, and may include the date, physician's name orother information unique to the procedure. This information may be handentered or preferably machine printed. Note that this information doesnot need to be visible in the emission spectra, because the visiblelight and emission spectra light images will be lined or merged,however, such visibility in the emission spectra is preferred. Onemethod of such printing is to use an ink-jet printer with fluorescentink.

Fiducial feature 11 permits a medical practitioner to add information tothe fiducial by hand at the time of the diagnostic procedure. Thisfeature allows the practitioner to add information desired by thepractitioner or relevant to the particular area of interest. Forexample, the practitioner may enter a mole number as the fiducial isapplied to each mole.

Fiducial features 10 and 11 are particularly valuable based on the waymany medical diagnostic procedures are performed. For example, in onepart of an office, clinic or hospital, the fiducial may be preparedusing feature 10 a few minutes or hours in advance, based on a scheduledappointment, along with other lab preparation. Then feature 11 is usedby the physician or technician immediately before or after the fiducialis applied to the patient. Thus feature 10 is most applicable to thescheduling of appointments and feature 11 is most applicable during theprocedure itself.

Fiducial feature 12 allows fiducial to be manufactured in advance. Eachfiducial may be provided with both standardized and unique information,such as a sequence number or lot number.

Fiducial feature 13 allows fiducials to have a combination of marks,some of which are visible in the emission band of the biotag and some ofwhich are visible in the visible light band. Because images of an areaof interest taken in visible light and in the emission light band aremerged, overlaid or linked, both types of marks will be visible anduseable in analyzing the diagnostic procedure results.

Note that some of these features may be combined into a single mark orgroup of marks. That is, a single mark on a fiducial may serve more thanone purpose.

The reference to “at least a pair” of marks may refer to two or moreportions of a single mark. For example, a single rectangle could serveas linear measure by using two sides of the rectangle. As anotherexample, a single circle could be used to align images by using morethan one portion of the circle for alignment.

In some embodiments, a fiducial may be on a single substrate, such astape or carrier, which may or may not stay with the fiducial when placedon the patient. Or multiple individual physical fiducial components maybe placed on the patient. One embodiment uses a donut shaped fiducialcarrier that surrounds the disease area of interest.

In one embodiment, fluorescent dye or compounds are placed within apolymer in the fiducial so that the dye or compound will not exit thepolymer or enter the patient's skin. The polymer may prevent degradationof the fiducial and may assist in the stabilization of the fluorescentcompounds. The polymer may prevent diffusion and assist in prevention ofa change of the calibration of the fiducial. The polymer may block strayor unnecessary light from entering the fiducial. The polymer may be acoating on the fiducial, or it may be integral with the fluorescentcompounds.

The fiducial may use more than one fluorescent dye or compound. In someembodiments, the dye or compound is not identical to a fluorescentmarker in the biotag.

Complex fiducials may be cut or modified during the diagnostic procedureto accommodate special locations. For example, a mole in the crease ofskin next to the nose may not accommodate a donut-shaped fiducial.Various shapes of fiducials may be created or selected for variouslocations on a subject's body.

A fiducial may be permanently implanted on the patient for long-termtracking.

Note that the shapes and arrangement of marks on the fiducials may varyconsiderably from the examples herein.

Optical System

A novel feature of this invention is the use of an integrated opticalsystem. The integrated optical system may be a consumer or prosumerdigital SLR camera, for example. By integrated, it is meant that thecamera body may include a power-supply such as a battery, an internalimage sensor, internal image storage memory, user controls convenientlyon the body, at least one user display, internal autofocus logic,internal control electronics including stored instructions for anembedded processor, and/or internal image processing logic includingstored instructions for an embedded processor. We refer to theintegrated optical system as a camera in this disclosure. Any discussionherein of a camera may apply to any integrated optical system and viceversa. The camera body is either attached to or includes anon-interchangeable lens, preferably a macro-lens, or the camera bodyaccepts interchangeable lenses. For this invention, a macro-focusinglens is preferable.

Prior art cameras are not integrated. That is, generally, the necessarycomponents and controls for operation are not contained in the body ofthe camera, and the camera is not manufactured in high volume. As such,they are rarely suitable for hand-held operation. They are alsoexpensive, as they are designed and built specially for a medicalapplication.

Modifying an “off-the-shelf,” or “consumer” camera for this specialpurpose medical application has several obvious benefits: the camera islow cost, reliable, self-contained, easily hand-held, and/or includeskey components such as a complete user interface, image display,auto-focus, and/or image storage. A key reason why such an approach hasnot been used before is that fluorescent biomarkers operate in theinfrared (IR) spectrum. Consumer cameras do not operate in the infraredfor at least one reason: the image sensor is covered with an IR filterto block IR light. The camera would not operate properly in the visiblespectrum without an IR filter. A second reason consumer cameras withphase-detection auto focus will not work in this application is that theautofocus sensor and algorithms work only with visible light, not withIR light.

It might be possible to configure a fixed-focus camera to work in the IRspectrum. However, a fixed focus camera with a reasonably high numericalaperture will have different focus points for visible and IR light. Tobe practical, in one embodiment, in this medical application, the camerapreferably takes two pictures of the target lesion: one in the visiblespectra one in the IR spectra looking at the emission from thebiomarker. The visible light spectral image is useful in order tocorrelate the glowing areas in the IR image with the exact area of skinon the patient. That is, the lesion needs to be accurately located. Itis also valuable to the physician to accurately compare what thephysician sees, that is, the visible light image, with what has beendetected as cancerous with the biomarker. This comparison is critical toanswer such questions as: (a) Are the visible lesion and cancerouslesion the same size and shape? (b) Is the cancerous portion of thislesion directly underneath the visible lesion? (c) Is only a part of thevisible lesion cancerous? (d) Has the cancerous lesion spread beyond thevisible lesion? (e) Are the cancerous lesion and the visible lesionseparate growths? (f) Is the signal from the lesion in question or oneon the periphery? Answers to such questions may aid in diagnosis.

A preferable size for detection can be below 25 mm in diameter and isnot limited to lesions above 5-6 mm in diameter. Lesions below 1, 2, 3,4, 5, or 6 mm in diameter can be imaged and/or analyzed.

Auto-focus becomes more critical or useful when: (a) the numericalaperture is larger, (b) the lens is closer to the subject, (c) themagnification is higher, or (d) the resolution is higher. Thecombination of these four factors, when implemented suitably for thisapplication, is such that autofocus becomes a practical necessity if thecamera is to image both visible and IR light. A medical camera usingvisible light for one image and IR for a second image, using a lens witha high numerical aperture, requires the use of autofocus because thefocus at the two different wavelengths will be different. The use of thecamera's built-in autofocus mechanism for these dual purposes is bothnovel and a major benefit of this invention.

There are two major types of autofocus used today, along with minorvariations. We describe each separately, and each of these two types ofautofocus is a separate embodiment of this invention. Various types ofautofocus can be incorporated alone or in combination with theinvention.

The first type of autofocus (AF) we describe we call contrast detection,although various terms exist in the art. Contrast detection ischaracterized by searching for the focal point that generates either thehighest spatial frequency components in the image, the most high-spatialfrequency components in the image, or the most contrast in the image, orsome combination or equivalent. The focus may be mechanically adjustedby moving the lens, moving an element within the optical path, movingthe image sensor, or by other means. This approach is most commonly usedin cameras with no mirror and/or using the image sensor for theautofocus, however, other implementations are possible. For example, amirror may be partially transparent.

Contrast detection autofocus is suitable for one embodiment of thisinvention with no changes to the autofocus algorithm or firmware, ormechanical focus mechanisms. However, some improvement may be possibleby changing either.

The second type of autofocus (AF) we describe we call phase detection,although various terms exist in the art. Phase detection ischaracterized by the use of an additional sensor besides the imagesensor, which has at least the function of autofocus: the AF sensor. Abeam splitter, and/or a partially reflective mirror, or other means isused to direct light from the subject to the AF sensor. Two micro-lensescapture the light rays coming from the opposite sides of the lens anddivert it to the AF sensor, creating a simple rangefinder with a basewithin the lens's diameter. The two images are then analyzed for similarlight intensity patterns (peaks and valleys) and the separation error iscalculated in order to find if the object is in front focus or backfocus position. This quickly gives the direction of focusing and amountof focus correction needed. This more complete information typicallyallows faster focusing than contrast detection.

However, when using phase detection AF in the IR, it is necessary tochange the firmware in the camera because the separation error isdifferent for IR than for visible light. Thus, in embodiments of thisinvention that use phase detection AF, the autofocus firmware ismodified to look for peak detection where the peak separation is in theemission band being used, rather than the peak separate for visiblelight. In the simplest case, this involves updating single constant inthe firmware.

For this invention, in the embodiment using phase detection AF, thecamera's internal IR blocking filter that is in the optical path of theAF sensor is removed.

Autofocus, when using light in the emission spectra of the biotag, mayeither be on the biotag itself, if present, or on the fiducials. The useof the fiducials assures proper autofocus, even if the biotag ismissing, weak or diffuse.

Spectra Considerations

Detection of the biomarker comprises exciting the detectable labelportion of the biomarker with light of an excitation wavelength, thenimaging the resultant longer emission wavelength light emitted by thelabel. Ideally, there is no overlap in the useful excitation spectra andthe useful emission spectra of the entire optical system. Any overlapwould cause some of the excitation light to be in the image, whereasideally no emission light would in the image. In some implementations,some overlap may occur.

As in all imaging, an important goal is to have a high signal to noiseratio. That is, have the most light from the target of interest, in thiscase cells with the biotag attached, and the least light from all othersources. In general, the brighter the excitation light, the brighter theemission light. Thus, one wishes to concentrate as much of theexcitation light in the most sensitive area of the excitation spectrum.A primary source of undesirable light is the excitation light beingpicked up in the emission photograph. Thus, one wants as little of theexcitation light as possible to be seen in the emission photograph. Bothof these goals are accomplished by specific elements of one embodimentof this invention, as described in detail below.

Another source of undesirable artifacts in the medical image isinconsistent illumination. Such lighting inconsistencies take manyforms, including vignetting or blotchy illumination. Theseinconsistencies make calibrated readings difficult or impossible.However, at the same time, one wishes to concentrate the energy used forexcitation light into the area of interest. Uniform illumination istypically at odds with such efficient illumination. Certain aspects ofsome embodiments of this invention optimize both of these goals, inparticular the design of the LED lighting sources and diffuser, as willbe explained in detail, below.

The “useful” excitation and emission spectra, including the final signalto noise ratio, depends on the end-to-end performance of the completeoptical system. The major elements to consider for the spectral analysisof the system include one or more of the following: illumination LEDdriver electronics; illumination LED(s); illumination filter; excitationspectra of the fluorescent compound(s); emission spectra of thefluorescent compound(s); emission filter; lens; IR filter (if any)covering the image sensor; image sensor; image processing. The shape ofemission spectra, filter spectrum, emission spectra, or sensitivityspectra for all components is called, simply, “spectra” herein. Thespectra for LED(s), and excitation and emission of the biomarker arefrequently peak shaped. The spectra for the filters are frequentlybox-shaped with steep sides. The spectra for the skin, lens(s) and imagesensor are more or less one-sided, with uneven, non-steep slopes on thedeclining side.

This invention includes but is not limited to novelty in the selection,positioning and implementation of specific components in the opticalchain, in order to achieve improved performance, cost reduction, andconvenience.

Major factors for each element in the optical chain that contribute tofinal image quality include spectra, mechanical alignment in all axis,and optical uniformity. The physical elements in the imaging chaincomprise the following, in nominal optical sequence:

Electronic drive for the illumination LED(s)

Illumination LED(s)

Illumination lens(s) and diffusers

Illumination filter

Subject skin

Subject lesion

Biomarker mechanical, spectral and optical performance

Emission filter

Imaging lens

IR filter (if any) over image sensor

Image sensor

Image processing electronics and algorithms

Any of these physical elements may be provided optionally. Additionalphysical elements may be provided. In some instances, the sequence ofone or more of the physical elements may be altered.

There are numerous other elements that have an impact on the final imagequality. Some of these include:

-   -   Scattered light in the optical system    -   Dust and other contaminants in the optical system    -   Alignment of optical components    -   Imperfect optical component, such as vignetting, distortion,        noise, absorption, internal reflections, and degradation over        time    -   Non-uniform illumination    -   Defects or variations in the image sensor    -   Mathematical weaknesses in the image processing algorithms    -   Inconsistencies of components due to manufacturing variations    -   Misalignment of the device by the operator    -   Motion of the device in use    -   Motion of the subject during exposure    -   Autofocus errors    -   Irregularities in the subject distance over the field of view    -   Image Viewing        A preferable embodiment uses a macro lens.

A preferable embodiment for viewing the visible light image and theemission light image is on a dynamic, electronic display, where the userinterface may include a slider or equivalent means to continuouslychange the image seen from the visible light image to the emission lightimage, and back, where the two images have been automatically aligned.

A preferable embodiment for delivery of automated melanoma detection isto match the features of the mole under review with features extractedfrom an image library using supervised learning. The mole underconsideration has its features measured automatically during imageprocessing. Currently, 28 features are considered out of over 300identified, including texture, size, etc. Any number of features may beconsidered during image processing, and the library may have any size.These extracted features are compared with the features previouslyextracted from the image library and classification is based on bestmatch. Classification is to provide images from an image library(“reference images”) that match as closely as possible the patient'smole or area of interest. The library images have previously beencharacterized, for example, by mole type and cancerous content, if any.In addition, a preferable embodiment provides one or more quantitativeassessments of how closely the patient images match the referenceimages. Ideally, but not necessary, these quantitative assessmentsrepresent a percent likelihood that the patient's area of interest isthe same mole type or cancer type (or disease) as the patient's area ofinterest.

In one embodiment a dynamic slider is used to compare two overlaidimages where one image is from the patient and the other image is areference image, presented either at the same effective resolution orsuch that the diameter of the mole or cancer is matched between the twoimages.

FIG. 1 provides a block diagram of a device used in accordance with anembodiment of the invention 14. Shown is the integrated imaging device1. The integrated imaging device may have a cavity for a memory card 2,which may include a wireless interface (not shown in the Figure), a userdisplay 3, and a user control 29. The user display can include a screenor other display that may show an image that may be captured by theintegrated imaging device. A lens may be provided or attached to theintegrated imaging device. The lens 13 is either integral to theintegrated imaging device or the device is adapted to acceptinterchangeable lenses and one such lens, ideally a macro lens, is showninstalled on the camera as 13. An operating button 4 may also beintegrated within the integrated imaging device. Other user interfacemechanisms such as touchscreens, levers, sliders, knobs or features maybe used for a user to interface with or interact with the integratedimaging device.

One or more filters may be provided in the integrated imaging device,attachable to the integrated imaging device, or can interact with theintegrated imaging device. The integrated device may have two filters, 5and 6 in a means, here shown as a slide 12, to move the filtersrespectively into the optical path of the camera. Any number of filters(e.g., 1, 2, 3, 4, 5 or more) may be provided. The filters may passdifferent wavelengths of electromagnetic radiation to pass through,relative to one another. The filters may be movable relative to theoptical path of the camera and/or one another. The filters may moveorthogonal to the optical path of the camera. Desired filters can beslid, pivoted, or rotated into place. Filter 5 is a visible band passfilter and filter 6 is a fluorescent emission band pass filter. A whitelight source 7, can be provided. The white light source may comprisewhite LEDs or any other light source. A fluorescent excitation lightsource 8, could comprise infrared LEDs. Two fluorescent excitation lightsources are shown in order to achieve uniform illumination of the moleor other target area. Uniform illumination is advantageous in achievinga calibrated or measurable response based on the biotag and/or thefiducials for this purpose. Fluorescent excitation band pass filters 9,may be provided between the fluorescent excitation light sources 8 andthe subject 11. The excitation band pass filters may be provided betweenthe excitation light sources and an area of interest or cavity and/ortissue surface of the subject. A structured light illumination component10, such as a diffuser may be provided, which may be integrated with oneor both white light sources 7 in order to achieve uniform white lightillumination of the subject. The diffuser may be an optical element thatmay diffuse or spread light.

Continuing with FIG. 1a , a light source 15 may be provided comprisingthe emission wavelength of the biotag. A narrow-pass-band filter 16 canbe used to restrict the light from 15 to just the emission wavelength,at least for the wavelengths sensitive to the camera. The combination ofthe light source 15 and filter 16 may be used in an autofocusembodiment, discussed elsewhere herein. The filter 16 may not berequired in all embodiments when the light source 15 is sufficientlynarrow-band. The light source 15 may be an LED, laser, fluorescentemitter, or other light source. Similarly, filter 9 may not be requiredin all embodiments when the light source 8 is sufficiently narrow band.8 may be LED, laser, fluorescent emitter, or other light source.

Note that the elements shown in FIG. 1a are not to scale and thearrangement of the elements as shown is purely exemplary. The number ofillumination elements may be two, as shown, or may be one element, ormore than two elements. Light directing elements such as mirrors,prisms, light-pipes, fiber optics or splitters may be used to direct thelight. Not all elements are required in all embodiments. In particular,the moving filters 5 and 6 in slide 12 may be required in someembodiments, as discussed in more detail elsewhere herein.

In one embodiment a single filter is used, instead of two. In thissingle-filter configuration, the filter has a band-reject notch at theexcitation frequency, such as 660 nm, while letting both visible andemission band light pass. In this way, such a single filter may be used,without changing filters, for both visible and emission exposures.

Structured illumination may be used to identify hair. The structuredillumination may also be used to determine the height and shape of themole above the surface of the skin, and the texture of the mole. Onetype of structured illumination is used to shine a series of a parallellight beams at a low angle to the skin. When photographed from an angleapproximately normal to the skin, the light beams will appear asparallel, straight on a flat surface, but will be distorted,non-straight, based on elevation and texture. Hair will be visible asmajor discontinuities in the parallel light beams. One method ofachieving structured illumination is with a diffuser with a series ofparallel slits in front of a white light source, through which areprojected parallel beams of light. A second method is to use an imageplate, which provides a series of brightly lit white lines, then use alens to image this image plate onto the skin. The image plate may be apiece of clear plastic with grooves machined in it, or an illuminatedplate overlaid with an opaque filter with transparent slits for thelines. A third method of achieving structured illumination is with aseries of parallel cylindrical lenses in front of a white light source.Structured illumination could be included, in some embodiments, as theshape of the plastic encapsulation over one or more LEDs. Yet anotherembodiment uses an interference pattern from laser light to create theparallel lines of light. One embodiment of structured illumination isshow in FIG. 12.

FIG. 1b provides a wire-frame view 20 of a system in accordance with anembodiment of the invention. Shown is the integrated imaging device 1,with an operating button 4. The user display 3 and memory card cavity 2are not visible in this view. Filters 5 and 6 are not visible in thisview. The filters 5 and 6 may be inside of the filter holder 21. Filters5 and 6 may be moved in or out of the optical path by activating a slide22. The white module 8 holds white LEDs, drive batteries, and/or thestructured light illumination component 10, not visible in this view.The fluorescent emission light module 7 also holds drive batteriesand/or the fluorescent band pass filter 9, not directly visible in thisview. A mounting ring 13 may be provided for an interchangeable lens.The mechanical components are held in position rigidly by a mountingplate 30. An attachment point 27 may be provided for a light baffle.This drawing shows two converging light sources, from 7 and 8, as twolight beams 24. These two light sources can illuminate the target area25 uniformly. Optionally, a diffuser or other optical elements may beprovided to assist with uniformity of illumination.

In some embodiments, a white LED 23 a may be provided at one end of aplastic fiber. The other end of the plastic fiber provides the whitelight 23 b to illuminate the area of interest 25. The fiber may be anoptical fiber capable of conveying light from a first end to a secondend.

Compartments 7 and 8 hold illumination LEDs, and optionally batteries orother power sources for the LEDs. Alternatively, power for theillumination light may be provided the battery in the integrated imagingdevice 1, or by a connector (not shown in FIG. 1b ) to an external powersource. Power may be provided in various combinations. Compartments 7and 8 may also hold other illumination sources such as elements 15 and16 shown in FIG. 1a . Compartments 7, 8 and/or 15 may be combined, ormissing entirely from some embodiments.

A ring 26 may be used for providing white light ring illumination. Astructural component 27 of the invention may be used as a camera-endtermination for a light baffle (not shown in FIG. 1b ).

No structured light illumination is shown in FIG. 1 b.

An alternative white light ring illuminator may be provided inaccordance with some embodiments of the invention.

FIG. 2a shows a block diagram of the steps in a method for use of oneembodiment for medical diagnosis. Note that all terms used are definedin this description. All steps are further defined and discussed withalternative embodiments elsewhere in this description. The medicalprofessional first places the biotag topically on the mole or other areaof interest 31. In alternate embodiments, the biotag can be injected orapplied to an area of interest in other manners. Any description of amole may apply to a lesion or other area of interest, or vice versa.After a short incubation period the medical professional removes theexcess biotag 32. The incubation period may be any predetermined periodof time. The medical professional places one or more fiducials close themole 33. The fiducial may be within an area of interest or adjacent toan area of interest. The fiducial may be proximal to an area ofinterest, for example within 30 mm, 25 mm, 20 mm, 15 mm, 12 mm, 10 mm, 8mm, 7 mm, 6 mm, 5 mm, 4 mm, 3 mm, 2 mm, or 1 mm of a mole, lesion orarea of interest. This step may be prior to steps 31 or 32, however theorder shown in this Figure is preferred. A user may then set the visiblemode of the camera, and place the camera in position (or the patient inposition) to block stray light, typically with the use of baffle andtake an image using visible light 34. The user may be a medicalprofessional. The medical professional then sets the camera tofluorescent mode and captures an image using the fluorescence of thebiotag, 35. 34 and 35 may be performed in the reverse order. Finally,all taken images from this patient are transferred out of the camera,preferably via wireless, 36. However, a memory module or wire mayalternatively be used to transfer images. Data from a camera may betransferred to one or more external devices. The data may be transferredwirelessly or via a wired connection. Data may be transferred directlyto the one or more external devices or over a network.

Often, a patient can have more than one mole. The entire process can berepeated for each area of interest on the patient. For convenience, step31 may be performed first for all moles, followed by 32 for all moles,then step 33 for all moles. Note that one capability of this inventionis the use of a combined fiducial to identify which mole is which on thepatient. Thus as step 33 is repeated the medical professional ideallyeither selects or writes on the fiducial prior to placement to identifythe mole. Preferably, all image for one patient step 36, is transferredat the same time. Clearly, for multiple areas of interest, steps may beperformed in various orders.

FIG. 2b shows a block diagram of steps in a method for computerizedimage analysis of the images taken in this invention. Step 41 startswith importing the images from the camera, ideally but not necessarilywirelessly. A computer, general purpose or specific purpose is used forsome or all of the steps in this Figure. A processor can be used forsome or all of the steps. Such a processor can be within the camera.Alternatively, it may on a PC, laptop, server, tablet, mobile device orin the internet cloud. Computerized image processing may be performed inthe camera, using its embedded processor, or on a computer, tablet,smart phone or other electronic computational device. The steps of imageprocessing are commonly split among multiple devices.

Instructions and data reside in computer readable media and/or memory.Step 42 uses the visible light image to locate the mole borders. Then,in step 43 the color and texture are extracted from the visible lightimage in within the determined mole borders. In steps 44 and 45 thelocator fiducials, as discussed elsewhere in this specification arelocated on both the visible and fluorescent image; these two steps maybe performed in either order, and may be performed prior to 41, 42, or43. Then, in step 46 the two images, the visible light and thefluorescent light, are aligned using the locator fiducials from theprior two steps. In step 47, mole features are extracted from one orboth images. This extraction is responsive to the known mole borders.For visible light images, which have been used traditionally to classifymoles, classification 48 is predominantly within the mole border.Supervised machine learning is performed on the library of images.Classification of moles is based on the features, or “characteristics,”learned from the library. Features calculated from the image underreview are compared to the distribution of features in the library.

Characteristics such as size, uniformity, texture and color are oftenconsidered. The biotag provides significantly improved diagnosticinformation, as the biotag is visible in the fluorescent image fordiseased cells only. The cancerous cells may extend beyond the border ofthe visible mole. Classification 48 against library images using thefluorescent image, or in combination with both images is likely toproduce more accurate diagnosis. Finally, the images are presented tothe medical professional 49. Ideally the visible light image and thefluorescent light image are presented as an overly, where the medicalprofessional, using a slider or similar means, can dynamically changethe overlay from 100% one image to 100% the other image as a way toeasily see how the two images align. Also, matching images from thelibrary are presented, along with quantitative matching coefficients andinformation about the library images. One or more of the steps describedherein may be optional, additional steps may be provided, or order ofthe steps may be altered.

FIG. 3a shows a cutaway view of the camera of this invention. Theintegrated imaging device body 1 contains a user-viewing screen 51.Plate 30 holds the camera components rigid. A primary optical path 61 ofthe camera may go through a lens 13. A primary optical path mayterminate at an area of interest. The two filters 5 and 6 previouslydescribed are shown visible in the filter holder 21. Filter 5 is in theoptical path in this drawing. The white light 7 and fluorescentexcitation light 8 modules can be included as previously described. Thelocations of the white LEDs 54 and the fluorescent excitation LEDs 60 intheir respective modules are shown. Fluorescent excitation band passfilter 9, previously described is shown. The structured illuminationcomponent 10, previously described, is shown. The paths of the whitelight 57 and 58 and fluorescent excitation light 59 and 24 can beprovided. The area of interest 25 may be provided. A compartment 56 forillumination batteries and power drive electronics for the illuminatorsmay be included.

FIG. 3b shows a perspective view of the camera. The integrated imagingdevice body 1 contains a user-viewing screen 51. Plate 30 holds thecamera components rigid. 13 is the lens. 21 holds the two filters 5 and6, not visible in this view. Filters are selected by the slide 22. 7 and8 are the white light module and the fluorescent excitation lightmodules, as previously described. 62 is the flexible light baffle. Theflexible light baffle may prevent undesirable light from entering thefield of view. The flexible light baffle may prevent ambient light fromreaching an area of interest, or reduce the amount of ambient light. Theflexible light baffle may be flexible to accommodate surfaces of varyingshapes or topologies.

One or more components described herein may be removable. For instance,one or more attachment having one or more filter and/or light source maybe added to a camera. The attachment may be permanently or removablyattached to the camera. In some instances, multiple levels or stages maybe provided that may be added to the camera.

FIG. 4 shows a visible light photograph of a mole. In this figure thephotograph is shown in black and white. The original photograph is incolor. Visible light photographs may be in black and white, color,monochromatic, or any other color scheme. 71 is a mole. 72 and 73 aretwo fiducials. These two fiducials can serve as both color referencesand as locator fiducials. Fiducials may have other features or uses asdescribed elsewhere herein. The fiducials may or may not be located at aknown distance from the moles. The fiducials may or may not have a knownsize.

FIG. 5 shows both a visible light photograph on the right andfluorescent photograph on the left of the same subject: a mouse withboth a cancerous and a non-cancerous lesion. 81 show the two fiducialsin both photographs serve both as locator fiducials to align the twoimages when overlaid and as biotag emission reference brightnessfiducials, as can be seen on the left image. 82 point so thenon-cancerous lesion in both photos. 83 points to the cancerous lesionin both images. As can be seen the non-cancerous image is nearlyinvisible in the fluorescent photo on the left.

Such images may be captured using the same device. For instance the samedevice can be used to capture a visible light photograph and afluorescent photograph. The same device can be used to capture aplurality of images, wherein at least some of the plurality of imageswere captured under different wavelengths of light. The images may beviewed separately and/or overlaid on one another. One or more of theimages may show a biotag in a visibly discernible manner. One or more ofthe images may not show the biotag in a visibly discernible manner.

FIG. 6a shows a fiducial 90 in accordance with some embodiments of thisinvention. Not all embodiments and not all features are necessarily usedin any one embodiment, application, device, method or use. The termfiducial can either refer to a physical object, such as printing, inkand die on a substrate, typically a plastic film suitable for placing onskin, or to a particular mark on that substrate. The fiducial mayoptionally have an adhesive or other feature that may permit it toattach to a surface, such as a subject's skin. Thus the singularfiducial and the plural fiducials are typically used interchangeably,subject to context. Here the substrate 91 is in the shape of donut,allowing a mole, lesion or other area of interest to be in the centerhole 92 of the physical substrate. The fiducial may be sized and/orshaped at least partially surround the area of interest. Many othershapes are possible, including individual dots, circles, ellipses,rectangles, or crescents. Fiducials are discussed in more detailelsewhere in this disclosure. A direction fiducial 101 providesanatomical orientation on the patient. An exposure and focus fiducial93, in this example as single area providing two functions, inconjunction with the other fiducials used on the patient for the sameexposure, provides sufficient area for auto-exposure setting by thecamera, and in this example provides a grid with many high-frequencyedges in at least axis for quality auto-focus by the camera. 94 and 99provide two scales for accurate measurement(s) of the mole or lesion.Note that in this example they are orthogonal. In some embodiments, itis advantageous if the camera or subject may be significantly non-normalto the area of interest. 95 provides a solid area for quantitativecalibration of the brightness of the biotag in the fluorescent image. 96consists of two locator fiducials that are used either manually orpreferable automatically to align the visible light and fluorescentlight images. 97 is an area or text to identify the patient and/ormedical professional and/or procedure. Depending on embodiment, thisarea is pre-printed during the manufacture of the fiducial; machineprinted at the office prior to imaging, or hand printed. 98 provides andarea in which the medical professional may handwrite. It is also an areato identify the particular mole on a patient with more than one area ofinterest. 100 provides medical tracking information such as manufacturerID, a LOT number and/or a sequence number. The sequence number may beused, in conjunction with medical records, to identify the procedure.Thus, this could be used an alternative to 97. In some cases fiducialmarks can be combined to provide more than one function.

One or more of the features described herein may be provided within afiducial. A fiducial may be a multi-function fiducial which may combinea plurality of the features discussed herein. A fiducial may be formedfrom a material that is not visible in a predetermined emission spectra.The fiducial may have one or more marks on the base that is formed fromsuch a material. The aggregate of all marks created at the original timeof manufacture of the fiducial may have a predetermined exposurebrightness in the predetermined emission spectra when exposed to lightin the predetermined excitation spectra. In FIG. 6b a machine-readablecode is shown 102. In this case the code is a QR code that contains thesame information as ID area 97 in FIG. 6a . Such a machine-readable codecould be used on a fiducial for automated medical records and as a wayto reduce errors in reduce costs, as a benefit. Any form of identifiermay be used. The identifier may be optically readable. The identifiermay emit a signal that may be read by another device. The signal may bea visible signal, RF signal, IR signal, wireless signal, or any othertype of signal.

FIG. 7 shows the relationship between various special bands used in oneembodiment of this invention. The horizontal axis shows wavelength innanometers and the vertical axis is percent from 0 to 100%. Curve 112 isthe excitation band for Cy 5.5 Fluorophore, showing excitationefficiency vs. wavelength. Curve 113 is the emission band for Cy 5.5Fluorophore, showing emission amplitude vs. wavelength. Both curves arenormalized with 100 at the peak. Curve 111 is the fluorescenceexcitation band pass filter spectral transmission v. wavelength, as usedin one embodiment. Curve 114 is the fluorescence emission band passfilter spectral transmission v. wavelength, as used in one embodiment.115 is the area of overlap of the curves 112 and 113.

Not shown in this Figure but relevant to the design and implementationare the spectral curves for the LEDs, lens optics, sensor, and imageprocessing.

FIGS. 8a and 8b show a benign mole topically treated with a biotag invisible light and IR light, respectively. FIGS. 9a and 9b show arecurring melanoma mole topically treated with a biotag in visible lightand IR light, respectively.

121 is the visible benign mole. 122 is the visible recurring melanomamole. FIG. 8b is almost completely dark, indicating no melanoma cells. Afaint border 124 is visible around the mole 123, which is the region ofthe skin on which the biotag was applied. Note that the mole 123 appearsdark over the faint area 124.

122 is the visible recurring melanoma mole. Note that the region aroundthe mole is indistinguishable from other normal skin on the patient. 125shows the same mole location under IR light. 126 shows the recurringmelanoma bright area around the same mole. Note that the visible portionof the mole also glows within the biotag region, rather than covering itdarkly, as in 123. Note that the recurring melanoma region 126 extendssignificantly past the border of the visible mole 122. Note that thetotal recurring melanoma area 126 of the patient's skin is visible inFIG. 9 b.

FIG. 10 shows an X-Y plot of two important features from 72 sampleimages. The X-axis is a texture features approximating the entropy ofthe mole area. The Y-axis is mean intensity of the biotag fluorescencein the area around the mole. The units shown on the graph are relativelyarbitrary units as a function of the specific image processingalgorithms used. These two features are two of 28 features automaticallydetermined by image processing of the images. Each of the 72 samples hasbeen medically classified into one of three groups: (a) melanoma, (b)dysplastic, or (c) nevus. There are 6 melanoma samples; 25 dysplasticsamples, and 41 nevus samples. The melanoma samples are shown asdiamonds; the dysplastic samples are shown as squares; the nevus samplesare shown triangles. As can be seen in the Figure, the nevus samples(triangles) tend to clump in the lower left; the dysplastic samples tendto clump in the center; and the melanoma samples in the upper right.

FIG. 11 is a flow chart of several embodiments of image acquisition,image processing through to mole classification. Each step is labeledwithin the box for that step and is discussed in detail previously. Step151 begins the sequence with the user pressing a button or equivalentoperation. Steps 152, 153 and 154 complete the acquisition of thevisible light image. The sequence then continues, depending on theembodiment, with steps.

One way to present data of this form to a medical professional is toshow the physician on a plot like this the specific patient samples ofinterest. Typically, the baseline of known samples would be much largerthan the 72 sample images seen here. The physician could then make hisher own judgment, based on the X-Y position of the patient's images onthe chart, of the relative risk to the patient, diagnosis and treatmentoptions. In another embodiment the invention provides a set of numericalmetrics to the physician representing either distance on the chart orcomputed likelihood that the patient sample is in one of thesecategories.

In one embodiment a series of areas, such as an ellipse, are placedaround each group of related moles. The areas represent probabilities,such as 50% or 90% that a mole of a particular type falls within thatarea. Then, for each patient image, a normalized metric is provided tothe physician representing the quality of fit for that patient imagewithin the most likely or most interesting areas. Thus, the medicalprofessional is provided with consistently produced metrics from theautomated image analysis, while the medical professional continues tomake decisions requiring medical judgment.

Of course many more relationships between the features are identified,typically using multi-variant analysis. Some of these relationships havehigher dimensionality than 2D (X v. Y). The scores for multiple featurerelationships may be aggregated to produce a small number of simplemetrics, such as the probability that a particular patient image isnevus, dysplastic or melanoma.

FIG. 12 shows an image of a mole 202 with structured illumination lines203. The skin around the mole is shown 201. Two or ideally more straightlines 203 are projected across the skin 201 and the mole 202. The lines203 are shown in this image as black, for clarity, although in apreferred embodiment they are white light, or monochromatic light suchas from an LED or laser. As the structured illumination lines 203 crossthe textured, raised, lowered, bumpy or mottled surface of the mole 202they deform 204 from straight. These deformations 204 show the relativeheight of that portion of the mole. The structured illumination lines203 are projected onto the mole 202 and skin 201 from an angle relativeto the angle of the camera to the mole, for the camera or optics thatare used to create this image. The known geometry of the illumination,camera and mole are used to compute the height (elevation) of the moleat each point of each line where it crosses the mole. Statisticalanalysis of these aggregated elevations is then used as part of theclassification algorithm, discussed elsewhere herein. For example,minimum, maximum, average, spacing of bumps, height of bumps, and othermetrics are readily computed from the aggregate elevations.

FIG. 13 shows an exemplary optical transmission curve for a singlefilter used in one embodiment. This filter employs a notch filter 302 atthe same wavelength as the emission light. The filter is a band pass forboth visible light 301, or most visible light, and for the emissionlight 303. The use of this single filter, rather than two filters, isdiscussed in the text above. Any number of filters may be employed withvarious band passes for various wavelengths. In some instances, nooverlap may be provided the transmitted wavelengths between thedifferent filters. Alternatively, some overlap may occur.

Another embodiment uses a third provided light source, rather thanfiducials, for the autofocusing step at the emission wavelength. In thisembodiment, rather than fiducials (or, in addition to fiducials), thearea of interest is illuminated by light in the emission spectra of thebiotag, such as by LEDs, or by a light source with a narrow-pass-bandfilter. The autofocus of the camera is then used to focus on the area ofinterest at this wavelength. Then, this “autofocus” light is turned off,the excitation-band light source is turned on, and the exposure istaken. This exposure comprises emission-band light from the biotag andis still in focus at this wavelength.

Gel formulations may comprise DMSO; Ethanol, 200 proof or PropyleneGlycol or Propylene Glycol or Glycerine; Hydroxypropyl cellulose, HF(Klucel) or Carbopol 980 or Carbopol 971 or carbomer; Trolamine. Forexample, a formulation may comprise DMSO 45 w/w, Glycerine 55.87 w/w,Carbopol 9801 w/w, Trolamine 0.13 w/w. In alternative formulations thesolvent is replaced with saline or a non-aqueous solution, e.g.MSM—methylsulfonylmethane. Alternative gelling agents include Methocelor Kucel, Carbopol 971 or carbomer.

It should be understood from the foregoing that, while particularimplementations have been illustrated and described, variousmodifications can be made thereto and are contemplated herein. It isalso not intended that the invention be limited by the specific examplesprovided within the specification. The descriptions and illustrations ofthe preferable embodiments herein are not meant to be construed in alimiting sense. Furthermore, it shall be understood that all aspects ofthe invention are not limited to the specific depictions, configurationsor relative proportions set forth herein which depend upon a variety ofconditions and variables. Various modifications in form and detail ofthe embodiments of the invention will be apparent to a person skilled inthe art. It is therefore contemplated that the invention shall alsocover any such modifications, variations and equivalents. It is intendedthat the following claims define the scope of the invention and thatmethods and structures within the scope of these claims and theirequivalents be covered thereby.

What is claimed is:
 1. A method comprising an imaging an area ofinterest comprising a first area and a second area on a surface of anindividual wherein the first area comprises a lesion, the imagingcomprising: a biotag to the area of interest; capturing a visible animage of the area of interest in a visible mode; capturing a fluorescentimage of the area of interest in a fluorescent mode; and obtainingfeatures of the lesion by combining the visible image and thefluorescent image.
 2. The method of claim 1, wherein the surface isselected from skin, cervix, oral mucosal surfaces, and bladder.
 3. Themethod of claim 1, wherein the biotag binds to the first area and/or thesecond area, and wherein the first area comprises tumor cells and thesecond area comprises a tumor macroenvironment.
 4. The method of claim1, further comprising providing a fluorescent fiducial to the area ofinterest.
 5. The method of claim 1 wherein the biotag comprises adetectable label that is detectable in a selected range of wavelengths.6. The method of claim 5 wherein the detectable label is a fluorescentdye.
 7. The method of claim 1, wherein the biotag comprises a firstfluorescence dye conjugated with one or more cancer sensitive biologicalentities selected from protein, RNA, and DNA.
 8. The method of claim 7,wherein the biotag comprises Cy5.5 as a detectable marker.
 9. A methodof claim 1 further comprising: illuminating the area of interest togenerate a structure illumination; autofocusing an integrated imagingdevice on the structure illumination; imaging the structure illuminationto calculate a three-dimensional image of the first area; illuminatingthe area of interest with a fluorescent excitation wavelength; imagingan fluorescent emission to obtain a fluorescent light image;illuminating the area of interest with white light; imaging in whitelight to obtain a white light image; measuring an attribute of the firstarea using automated image processing of a three-dimensional imageobtained from the white light image and the fluorescent light image;inputting the measurement as input to software; and detecting acancerous or pre-cancerous lesion from the three-dimensional image. 10.The method of claim 9, wherein the integrated imaging device comprises:an image sensor, an autofocus control logic, an embedded processor withmemory for stored instructions and one or more of: (i) a lens, or (ii) alens mount adapted to accept a lens; such that a field of view of theintegrated imaging device includes the area of interest.
 11. The methodof claim 10, wherein the step of imaging fluorescent emission comprises:activating a user control on the integrated imaging device to initiatean imaging sequence, where the imaging sequence comprises: illuminatingthe area of interest by an emission light source in an optical band thatoverlaps with an emission optical band of the biotag; autofocusing bythe autofocus control logic within the integrated imaging device on thefield of view and turn off autofocus illumination while maintaining afocus position and illuminating the area of interest by an excitationlight source in an optical band that overlaps an excitation optical bandof the biotag; and exposing the integrated imaging device to the fieldof view.
 12. The method of claim 1, further comprising: placing afiducial next to the lesion; and detecting the fiducial in athree-dimensional image and registering with the visible image and thefluorescent images.
 13. The method of claim 1, wherein the step ofcapturing an image of the area of interest comprises: placing anintegrated imaging device comprising: an internal image sensor, internalautofocus control logic, internal embedded processor with memory forstored instructions and one or more of: (i) a lens, or (ii) a lens mountadapted to accept a lens; such that a field of view of the integratedimaging device includes the first area and the second area of the areaof interest; activating a user control on the integrated imaging deviceto initiate a first imaging sequence, where the first imaging sequencecomprises: illuminating the area of interest by an excitation lightsource in an optical band that overlaps an excitation optical band ofthe biotag; autofocusing by the autofocus control logic within theintegrated imaging device on the field of view; and exposing theintegrated imaging device to the field of view.
 14. The method of claim13 wherein the fiducial comprises at least one mark used as a focustarget in autofocusing.
 15. The method of claim 13, further comprising:determining an auto-exposure setting, wherein the fiducial furthercomprises at least one calibrated exposure mark, and wherein theauto-exposure setting uses the at least one calibrated exposure mark;automatically numerically measuring a quantity of the biotag in the areaof interest using the at least one calibrated exposure mark; andpresenting to a medical service a numerical measurement of the quantityof the biotag at a same time as at least one of the images produced bythe first and second imaging sequences.
 16. The method of claim 1,wherein the fiducial comprises: a base that is not visible in apredetermined emission spectra when exposed to light in a predeterminedexcitation spectra comprising: a fluorescent calibration mark with apredetermined calibration brightness in the predetermined emissionspectra when exposed to light in a predetermined excitation lightspectra, wherein this mark comprises a solid area, a fluorescent focusmark with sharply defined borders adapted for use by an autofocusmechanism of an integrated optical system; a pair of measurement markswith predetermined spacing; a fluorescent alignment mark visible in bothvisible light and visible in light in the predetermined emission spectrawhen exposed to light in the predetermined excitation spectra whereinthis mark is not circularly symmetric; and a direction mark adapted toidentify at least one anatomical term of location for a patient.
 17. Themethod of claim 1, wherein the lesion comprises a cavity surface or atissue surface comprising a lesion suspected to be cancerous orpre-cancerous and the second area comprises an area surrounding thefirst area.
 18. The method of claim 1, further comprises segmenting thevisible image to find a border of the first area.
 19. The method ofclaim 18, further comprises extracting a color and a texture from thevisible image.
 20. The method of claim 1, wherein a fiducial locatorcombines the visible image and the fluorescent image.
 21. The method ofclaim 1, further comprises an imaging of the area of interest in anemission spectrum before application of the biotag in the first area.22. A method of imaging an area of interest on skin of an individual themethod comprising: applying topically to the skin, in an area ofinterest comprising a lesion suspected to be cancerous or pre-cancerousa specific binding partner that binds to a targeted molecule of interestwherein the specific binding partner is a polypeptide of less than10.000 daltons; capturing a visible image of the area of interest in avisible mode; capturing a fluorescent image of the area of interest in afluorescent mode; obtaining features of the lesion by combining thevisible image and the fluorescent image.
 23. The method of claim 22,further comprising: placing a fiducial next to the lesion; and detectingthe fiducial in a three-dimensional image and using fiducial dimensionsto correct for magnification effects between the visible image, thefluorescent image and the three-dimensional image.